Skip to content

feat(git-providers): the agent's and the thread's repositories become references - #7035

Merged
viktormarinho merged 2 commits into
mainfrom
t3code/git-providers-repository-refs
Sep 8, 2026
Merged

feat(git-providers): the agent's and the thread's repositories become references#7035
viktormarinho merged 2 commits into
mainfrom
t3code/git-providers-repository-refs

Conversation

@viktormarinho

@viktormarinho viktormarinho commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Stacked on #7022. The expand half of moving the last two repository bindings
out of JSON.

Migration 201 gave task_board_items, task_board_item_prs and
org_repo_sync a real repository_id, and left the two that matter most to a
person using the product:

  • an agent's repository — connections.metadata.githubRepo, since a
    virtual MCP is a connections row;
  • a thread's extra checkouts — threads.metadata.githubRepos, the list
    TASK_ADD_REPO appends to so one run can hold several.

Why it's worth a migration

  • Integrity. The other three have ON DELETE SET NULL. Delete a repository
    today and every agent pointing at it via JSON silently points at nothing.
  • It retires a per-request lookup. Step 2 of resolveRepoTarget
    findByRef on a lowercased owner/name — exists only because the binding
    carries no id. It is a join standing in for a foreign key.
  • The thread list loses a real bug. Its append is a jsonb_agg rebuild
    inside one UPDATE, written that way because two concurrent TASK_ADD_REPO
    calls lost each other under read-modify-write — with the pod already holding
    the checkout the lost entry described, so nothing looked wrong until the pod
    was recreated without it. A primary key makes that ON CONFLICT DO NOTHING.
    Its dedup key was also lower(owner/name) with no host, so two
    acme/site on different hosts collided — the same class fixed in
    repoKeyFromCloneUrl in feat(git-providers): first-class repositories behind a provider interface #6939. A reference cannot collide.

Shape

ALTER TABLE connections ADD COLUMN repository_id text
  REFERENCES repositories(id) ON DELETE SET NULL;     -- SET NULL, like 201's

CREATE TABLE thread_repositories (                     -- CASCADE both sides:
  thread_id, organization_id, repository_id, added_at, --   a checkout means
  PRIMARY KEY (thread_id, repository_id)               --   nothing without
);                                                     --   either end

githubRepos is 1:N with insertion order and dedup by identity, so it is a
join table; the agent's is 1:1 nullable, so it is a column. (The plural list
lives on the thread, not the agent — worth stating, because the names
invite the opposite reading.)

Expand only — this changes no behaviour

Both bindings are dual-written, JSON included. During a rolling deploy a
pod on the previous release still reads only the JSON, and for the same reason
the JSON — not the reference — is still the more complete source while both
versions write. Preferring the reference today would lose repos an old pod
added mid-deploy.

So: this release writes both. The next flips the read to
reference ?? JSON and drops resolveRepoTarget's identity step. A third
drops the JSON.

The backfill only links — 201 already created a repositories row for every
agent binding (its fourth insert). Threads it skipped deliberately, so the rows
a checkout list names are created here first, anonymous like every other
identity-only source.

Sized against prod before writing

311 agents carry a binding, out of 11,575 connections; 29 threads carry a
checkout list, out of 95,678. Both backfills are a rounding error, and the
connections index is partial for the same reason.

Testing

Real-Postgres, because the whole migration is SQL against two differently
shaped metadata columns (connections.metadata is TEXT, threads.metadata is
jsonb) and an in-memory fake would agree with a version that links nothing:
every JSON binding ends up referenced, an agent with none stays null, a
repository only a thread names gets created, a connection whose metadata is
not JSON does not abort the run (the guard 201 needed), the backfill is
idempotent, and the JSON is left intact.

check, fmt, lint, knip clean.


Summary by cubic

Makes the agent's and thread's repository bindings real foreign keys instead of JSON metadata, without changing behavior yet (expand-only).

  • Adds repository_id to connections (agents) and a new thread_repositories table with a composite primary key.
  • Dual-writes both references and the existing JSON for backward compatibility during rolling deploys.
  • Fixes lost updates in TASK_ADD_REPO concurrent calls and dedup collisions across hosts.
  • Backfills existing bindings and creates repository rows for thread-only checkouts.
  • Raises the migration-test timeout to 30s; the per-test schema reset truncates every table and outgrew bun's 5s default on loaded CI runners.
  • Verified with real-Postgres integration tests; no migration steps needed for this release.

Written for commit 5b257f5. Summary will update on new commits.

Review in cubic

@viktormarinho
viktormarinho force-pushed the t3code/git-providers-change-requests branch from 85b758e to d94f3ea Compare September 8, 2026 17:45
Base automatically changed from t3code/git-providers-change-requests to main September 8, 2026 19:16
… references

Migration 201 gave three consumers a real `repository_id` and left the two
bindings that matter most to a person using the product still in JSON: the
repository an AGENT works in (`connections.metadata.githubRepo` — a virtual MCP
is a connections row) and the extra checkouts a THREAD holds
(`threads.metadata.githubRepos`, what `TASK_ADD_REPO` appends to).

That cost three things. Deleting a repository left every agent pointing at
nothing with no FK to say so. "Which agents use this repository" was a JSON
scan. And `resolveRepoTarget` had to keep a resolve-by-identity step — a
lowercase path match, per request — purely because the binding carried no id.

The thread list also loses a real bug. Its append is a `jsonb_agg` rebuild
inside one UPDATE, written that way because two concurrent `TASK_ADD_REPO`
calls lost each other under read-modify-write — with the pod already holding
the checkout the lost entry described, so nothing looked wrong until the pod
was recreated without it. A primary key makes that an `ON CONFLICT DO NOTHING`.
Its dedup key was also `lower(owner/name)` with no host, so two `acme/site` on
different hosts collided; a reference to a row cannot.

EXPAND ONLY, and deliberately so: this changes no behaviour. Both bindings are
dual-written, JSON included, because during a rolling deploy a pod on the
previous release still reads only the JSON — and for the same reason the JSON,
not the reference, is still the more complete source while both versions write.
The read flip and the JSON removal are the next two releases.

The backfill only links: 201 already created a repositories row for every agent
binding. Threads it skipped, so the rows a checkout list names are created here
first, anonymous like every other identity-only source.

Sized against prod before writing: 311 agents carry a binding out of 11,575
connections, and 29 threads out of 95,678 carry a checkout list. Both backfills
are a rounding error; the `connections` column is indexed partially for the
same reason.

Covered by a real-Postgres test, because the whole migration is SQL against two
differently shaped metadata columns (`connections.metadata` is TEXT, `threads`
is jsonb) and an in-memory fake would agree with a version that links nothing:
every JSON binding ends up referenced, a repository only a thread names gets
created, a connection whose metadata is not JSON does not abort the run, the
backfill is idempotent, and the JSON is left intact.
@viktormarinho
viktormarinho force-pushed the t3code/git-providers-repository-refs branch from 1f8cddf to c01e74d Compare September 8, 2026 19:30
Each of these tests resets the schema in a per-test hook, and that reset
truncates every table in it — a cost that grows with every migration the
repo adds. They already ran ~2s against bun's 5s default; on a loaded CI
runner the hook outlasts it, and bun aborting a hook mid-flight surfaces
as an unrelated "driver has already been destroyed" from the teardown
racing the seed. Give the hook a ceiling that matches what it actually
does.
@viktormarinho
viktormarinho merged commit 749b3bd into main Sep 8, 2026
33 checks passed
@viktormarinho
viktormarinho deleted the t3code/git-providers-repository-refs branch September 8, 2026 19:51
decocms Bot pushed a commit that referenced this pull request Sep 8, 2026
PR: #7035 feat(git-providers): the agent's and the thread's repositories become references
Bump type: minor

- decocms (apps/api/package.json): 4.339.1 -> 4.340.0
- @decocms/native (apps/native/package.json): 4.339.1 -> 4.340.0

Deploy-Scope: server
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant