Skip to content

perf(grouping): return file indices instead of paths - #1209

Merged
lizhengfeng101 merged 2 commits into
mainfrom
perf/grouping-file-indices
Sep 12, 2026
Merged

lizhengfeng101 merged 2 commits into
mainfrom
perf/grouping-file-indices

Conversation

@lizhengfeng101

Copy link
Copy Markdown
Contributor

Why

The grouping LLM call asked the model to echo full file paths back in its JSON response, so the output size grew with the sum of all path lengths. On large change sets this overflowed the completion token limit; the truncated JSON then failed to parse and the whole change set degraded to per-file dispatch, multiplying downstream review calls. This is the main reason grouping was slow on big diffs.

What

Backend — grouping contract switched from paths to integer indices

  • buildFileList prefixes each file with a zero-based index, e.g. [0] MODIFIED path (+12/-3).
  • groupingResponse.Files is now []int; the model returns those indices, e.g. [{"label":"...","files":[0,1]}].
  • parseGroupingResponse maps indices back to diffs by position, skipping out-of-range indices (the index equivalent of the previous unknown-path skip) and duplicates. A parse failure still returns an error and the caller falls back to per-file dispatch, exactly as before.
  • Prompts updated to ask for integer indices.

The response is now an order of magnitude smaller, so truncation on large change sets becomes rare instead of common.

Viewer — resolve indices back to paths for display

Because the grouping response is now indices, the session viewer renders them as paths:

  • buildGroupingIndex scans the request's numbered file list (user message only) to build an index→path map.
  • parseGroupingGroups unmarshals the response into indices; a legacy path-string response reports not-ok so the viewer keeps showing the raw text (already readable for those older sessions).
  • groupingView maps indices to paths and falls back to the raw response when nothing resolves (format drift), avoiding a wall of #idx.
  • The grouping card renders label + resolved paths with a collapsible raw response for audit.

No on-disk format changes; existing sessions render retroactively.

Scope notes

  • No streaming/truncation-salvage. Indices already shrink the output ~10x; a truncation that still happens falls back to per-file dispatch, the same as the path version did. Kept intentionally out of scope.
  • No lenient int decoding. Prompts require integer indices; an occasional format drift falls back to per-file (existing behavior). Left as a possible future hardening if real usage shows it is needed.

Testing

  • go build ./... and go vet ./... pass.
  • internal/agent and internal/viewer test suites pass, covering index parsing, out-of-range/duplicate handling, index→path resolution, format-drift fallback, and end-to-end viewer rendering.

The grouping LLM call asked the model to echo full file paths back in its
JSON response, making the output size proportional to the sum of all path
lengths. On large change sets this overflowed the completion token limit;
the truncated JSON then failed to parse and the whole change set degraded to
per-file dispatch, multiplying downstream review calls.

Switch the grouping contract to integer indices:

- buildFileList prefixes each file with a zero-based index, e.g.
  "[0] MODIFIED path (+12/-3)".
- groupingResponse.Files is now []int; the model returns those indices.
- parseGroupingResponse maps indices back to diffs by position, skipping
  out-of-range indices (the index equivalent of the previous unknown-path
  skip) and duplicates. A parse failure still returns an error and the
  caller falls back to per-file dispatch, exactly as before.
- Prompts updated to ask for integer indices.

The response is now an order of magnitude smaller, so truncation on large
change sets becomes rare instead of common.

Because the grouping response is now indices, the session viewer resolves
them back to paths for display:

- buildGroupingIndex scans the request's numbered file list (user message
  only) to build an index->path map.
- parseGroupingGroups unmarshals the response into indices; a legacy
  path-string response reports not-ok so the viewer keeps showing the raw
  text (already readable for those older sessions).
- groupingView maps indices to paths and falls back to the raw response
  when nothing resolves (format drift), avoiding a wall of "#idx".
- The grouping card renders label + resolved paths with a collapsible raw
  response for audit. No on-disk format changes; existing sessions render
  retroactively.
@github-actions

Copy link
Copy Markdown
Contributor

OpenCodeReview: Review complete: 0 finding(s) across 5 selected item(s).

…dices

# Conflicts:
#	internal/viewer/store.go
@lizhengfeng101
lizhengfeng101 merged commit 494bf1c into main Sep 12, 2026
12 checks passed
@lizhengfeng101
lizhengfeng101 deleted the perf/grouping-file-indices branch September 12, 2026 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants