Skip to content

feat(relay): sync artifacts with a time overlap - #8523

Open
dr0uleau wants to merge 12 commits into
mainfrom
drouleau/artifact-snapshots
Open

dr0uleau wants to merge 12 commits into
mainfrom
drouleau/artifact-snapshots

Conversation

@dr0uleau

@dr0uleau dr0uleau commented Oct 21, 2026 •

Copy link
Copy Markdown

Summary

Buzz clients need to recover task changes after a disconnect. This adds reads of changed artifacts with database modification timestamps and a 120-second overlap. Each result contains the current revision or an explicit removal. A separate read of known IDs confirms accepted edits and restores access state.

Pages use a timestamp and a stable ID to handle equal timestamps. Access grants include older records. Access policy changes require discovery and revalidation of saved IDs. The reader does not load a list of all readable channels. Ordinary artifact writes update their own rows.

Why the new timestamps are needed

created_at comes from the event signer. received_at records event arrival. Neither advances when access changes or an event is redacted. Sync needs database timestamps for those changes:

Column Why sync needs it
artifact_heads.modified_at Finds current records that change through an edit, deletion, or redaction.
artifact_revisions.modified_at Dates retained move records. This lets sync remove a task from its former channel when the reader cannot access its new channel.
channel_members.sync_modified_at Finds grants, rejoins, and removals. A grant must include older tasks whose own timestamps do not change.
communities.artifact_policy_modified_at Requests recovery after a relevant channel visibility, TTL, or deletion change. Clients revalidate saved IDs. Ordinary task writes do not update it.

The database sets these timestamps with clock_timestamp(). The 120-second overlap covers the accepted delay between timestamp assignment and commit visibility.

BUZZ_ARTIFACT_SYNC_ENABLED defaults to off. Apply migrations 0068–0070 and upgrade all writers before enablement. The 120-second delay bound is an operating assumption. Deployment steps cover index prebuilds and rollback.

Related issue

No duplicate found. Related work: read permissions, statement limits, and atomic channel settings.

Testing

A local signed HTTP test reads 10,050 tasks across 60,002 memberships. It uses a second relay state with the same signing key. The initial read takes 9.31 seconds over 101 pages with 4,923-byte events. A later read with no changes takes 7.97 milliseconds. These results cover local small-body reads.

cargo test -p buzz-relay sync_downloads_ten_thousand_tasks_with_sixty_thousand_memberships -- --ignored --nocapture

The protocol defines progress, removals, limits, and recovery. Relay verification covers the signed HTTP path. Native client verification belongs to the client PR.

Generated with Codex

Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
…pshots

* origin/main:
  feat(relay): add announce channels with a join_role setting (#8143)

Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
…pshots

* origin/main:
  fix(db): bound every statement and the #p mention scan (#8461)

Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
@dr0uleau
dr0uleau marked this pull request as ready for review October 22, 2026 00:11
@dr0uleau
dr0uleau requested a review from a team as a code owner October 22, 2026 00:11
@dr0uleau dr0uleau changed the title feat(relay): add consistent artifact snapshots feat(relay): sync artifacts with a time overlap Oct 22, 2026
@dr0uleau
dr0uleau deployed to codex-review October 22, 2026 00:17 — with GitHub Actions Active
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 22, 2026
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Status: review required for the current range.

The current range is 347ce88d5dd8bc631c073f1947fea6b75d5a43c0...4019c1df8d266701de685f9c714f747c1781967a.
A new review must complete for this exact range. When manual authorization
is required, a user with write access must comment exactly
@buzz-security-review 4019c1df8d266701de685f9c714f747c1781967a to authorize a new review.
Any previous review applies only to its recorded range.

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account.

One P2 correctness finding is inline: a concurrent first artifact can escape the conditional policy reset when its channel becomes private or is deleted. No code changes were applied.

Reviewed head 6a52195bd9a387e4871e3689b5c942ae42a5d597 against base/merge-base 2f2f6e2774fb0899d2a927f5e5b1987603765f31, with independent Rocket (systems/database), Mantis (correctness/security/concurrency), and Groot (protocol/user flows) lanes plus my integration.

Source-only review; no tests, builds, SQL or app execution. A single CI snapshot showed 50 successful, 28 skipped and 1 in-progress checks. Production query costs, migrations/rollback, client recovery and the documented 120-second stamp-to-visibility assumption remain unverified. Matching TTL artifacts intentionally return 422; clients must account for that permanent-channel-only scope.

Optional documentation clarification: spell out applying 0068 after its index prebuild, then 0069, before building the 0070 indexes; the current runbook jumps from the 0068 prebuild to 0069. Public PR description inspected; no attached images or actionable disclosure found.

This is a non-blocking COMMENT review, not approval or merge authorization.

Comment thread crates/buzz-db/src/store/channel.rs Outdated
@dr0uleau
dr0uleau requested a review from a team October 22, 2026 00:41
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
…pshots

* origin/main:
  fix(db): keep pgschema re-apply from dropping runtime partitions (#8487)

Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 22, 2026
@dr0uleau
dr0uleau deployed to codex-review October 22, 2026 01:10 — with GitHub Actions Active
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 22, 2026

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source follow-up via Wes’s account.

The prior P2 is fixed: actual visibility/TTL changes and successful channel deletion now advance the policy stamp even when the channel has no artifacts. The first-artifact interleaving test covers both transitions, and the deployment instructions now include 0068 application and migration-ledger recording.

One P2 remains inline, independently confirmed from the new security-review evidence: a continuation can expose an inaccessible destination revision ID. This is separate from the repaired policy-stamp race.

Reviewed head 357da40b3941efd083ea053fab56dbef3cce619c against base/merge-base 347ce88d5dd8bc631c073f1947fea6b75d5a43c0, with independent Rocket (systems/database), Mantis (correctness/security), and Groot (protocol/user flows) lanes plus my integration. Range-diff preserves all six previously reviewed feature patches; this pass covers the follow-up changes and the newly evidenced cursor issue.

Source only: no tests, builds, SQL or apps executed. One CI snapshot showed 29 successful, 21 skipped and 5 in-progress checks. Runtime interleavings, rollout/rollback, reset costs, client recovery and the 120-second visibility assumption remain unverified. Public description inspected; no attached images or actionable disclosure there. Non-blocking COMMENT review; no approval or merge authorization.

Comment thread crates/buzz-db/src/store/artifact_sync.rs Outdated
@dr0uleau
dr0uleau requested a review from a team October 22, 2026 01:27
Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 22, 2026

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source follow-up via Wes’s account.

The prior P2 cursor disclosure is fixed. Move-source continuation positions now contain artifact IDs instead of inaccessible destination revision IDs; DISTINCT preserves unique equal-timestamp positions. The current-ID read retains the caller’s query predicates and intersects them with the requested artifact IDs.

No new actionable findings in the follow-up from 357da40b3941efd083ea053fab56dbef3cce619c to reviewed head 4019c1df8d266701de685f9c714f747c1781967a, against base/merge-base 347ce88d5dd8bc631c073f1947fea6b75d5a43c0. Rocket, Mantis, and Groot independently reviewed systems/pagination, correctness/security, and product/protocol behavior; Star Lord integrated the results. The public description and changed commit/source material had no actionable disclosure finding; the description contains no attached images.

Source only: no tests, SQL, builds, or app workflows were executed. The new tests exercise the production HTTP router with fixture identity headers and direct database ingestion; this review does not establish signed-HTTP authentication coverage. CI snapshot: 43 successful checks, 28 skipped, 3 still running. Query-plan cost of DISTINCT, production latency/concurrency, client recovery and native accessibility/focus behavior remain unverified. The documented disabled-by-default rollout gate and 120-second overlap assumption remain applicable. This COMMENT is not approval or merge authorization.

@dr0uleau
dr0uleau requested a review from a team October 22, 2026 02:07

This branch was successfully deployed

No deployments
codex-review — 4019c1df Deployed Oct 22, 2026 by dr0uleau via Run Codex Security Review #9439
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants