Skip to content

feat(read-state): count thread replies only in followed threads - #8238

Closed
baxen wants to merge 1 commit into
mainfrom
honey/read-follows
Closed

baxen wants to merge 1 commit into
mainfrom
honey/read-follows

Conversation

@baxen

@baxen baxen commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

What

The sidebar now counts thread replies only in threads you follow, and ingest records who follows. Before, every sidebar request worked this out by looking up your "conversations" (who wrote each reply's parent) and scanning up to 4096 events backward per channel. Now:

  • Ingest writes follows. Storing an eligible channel message, in the same transaction:
    • The author is marked read through it. That's the timeline for a top-level post, or the thread for a reply, which the author then follows.
    • Each mentioned channel member follows the thread. For a top-level post, that's the thread rooted at it.
    • On a reply, the root's author follows too, and in a DM every member does.
    • New followers start just before the message, so it's unread to them. Existing followers keep their position.
  • The sidebar counts forward from each read position. A caught-up reader costs a few index probes per channel and per followed thread. A reader who's behind walks only what arrived after their position. Counts stop at 100.
  • Reading never follows. A thread mark_through moves an existing follow and otherwise does nothing.
  • participation.rs (the per-request conversation lookup) is deleted.

No migration, no new index, no wire change: the #8220 response shape is unchanged.

Behavior changes

  • A broadcast=1 reply counts only for people who follow its thread. A top-level broadcast=1 is no longer a mention.
  • A thread counts if you posted anywhere in it (root included), were mentioned in it, or are in its DM. Main counted replies to your own messages; now any reply in a thread you follow counts.
  • Mentions on channel rows come from event_mentions (valid p tags). Malformed tag documents no longer exclude a message.
  • A message backdated more than 15 minutes before the reader's position isn't counted. That's the relay's accepted author-time skew.
  • mentions on channel and thread rows is capped at 100.
  • A channel row is unread exactly when its latest_id is someone else's and unread, so a badge always has an anchor that clears it. A channel whose newest 4096 events are all reactions, edits or replies shows no badge.
  • No backfill: follows start when this relay is deployed.

Benchmark

Same harness as #8188/#8220 (a bench_unread Postgres test, not committed): one community, 20 channels, 8 other members. The scenarios are a caught-up reader with 5000 followed threads, a 20k-message timeline backlog with 12 mentions, 500 unread threads, a 20k-reply burst thread, and a 20k-reply thread spread over five days. Release build, local Postgres 14. Main (326e230) and this branch ran back to back. Sidebar numbers are p50 over 30 requests.

Scenario main this PR Counts (main → PR)
Caught up, 5000 followed threads 8.1ms 13.4ms nothing unread on either
20k timeline backlog 27.4ms 11.7ms mentions 2 → 12 (12 were sent)
+ 500 unread threads 38.1ms 16.4ms 25 thread rows on both
Burst thread, 20k replies in 10 min 18.7ms 11.2ms 4096 → 100 (cap)
+ far-behind burst thread 72.0ms 29.4ms
Spread thread, 20k replies over 5 days 20.2ms 1.0ms 4095 → 100 (cap)

Main undercounts the backlog's mentions: its backward walk stops 4096 events from the top, before the older mentions. This PR walks forward from the read position, so it finds all 12.

Being caught up costs more here, because this PR does a few index probes per followed thread and main does one walk per channel. 13ms for 5000 followed threads is still small.

Ingest pays for it. Each stored eligible message now writes the author's read position and any new follows in the same transaction:

Ingest main this PR
Top-level, serial, after ANALYZE (p50) 0.31ms 0.49ms
Same channel, 16 concurrent writers 9576/s 9294/s
16 channels, concurrent 12134/s 10365/s
Mentioning replies in 16 threads, concurrent 7329/s 6136/s

That's roughly +0.2ms per message and 3–16% less concurrent throughput. Most of it is the per-row write-fence trigger and foreign-key checks on the frontier and account tables. The serial runs before the first ANALYZE (0.91 vs 0.32ms top-level, 1.44 vs 0.50ms reply) run against never-analyzed read-state tables and overstate the cost. Autovacuum analyzes those tables in production.

Tests

  • New follows_postgres_tests.rs (12 tests): who follows on a reply, a mention (members only, starting just before it), a later mention, the root author, a top-level mention, DM members, a nested reply, and a message stored without thread metadata. Also: an unfollowed or broadcast thread never counts and marking it follows nothing, posting reads the timeline, the thread cap, and ingest never starting an account.
  • arrival_postgres_tests.rs: a late arrival behind 300 reactions clears on mark, and a message behind a full 4096-event budget of reactions shows no badge, because it has no anchor to mark.
  • projection_postgres_tests.rs: mentions are valid p tags in a stream and every message in a DM; a top-level broadcast is not a mention.
  • Existing sidebar, thread, arrival and relay buzz_v1 tests updated for following.
  • personal_read + buzz_v1 Postgres tests: 51/51 pass. Full buzz-db + buzz-relay Postgres run: 858 pass. The 25 failures (24 partition tests and one huddle JSON test) are local PG14 environment issues and fail the same way on main. cargo clippy -D warnings and cargo fmt are clean.

🤖 Generated with Claude Code

Ingest records follows in the message's transaction: the author reads
through their own message, and mentioned channel members, the root's
author and DM members follow the thread just before it. The sidebar then
counts forward from each read position instead of resolving
participation and walking 4096 events back per channel, and counts stop
at 100. A thread mark_through moves an existing follow and never
creates one. participation.rs is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Honey <8e307ae0076a4dab6b94b036ea3edc7e08f823a625269c1e6919e881a048b4d2@buzz.block.builderlab.xyz>
@baxen

baxen commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Superseded by #8264. That PR carries this change together with follow intents, the threads view and the arrival-ordered access paths, as one review.

@baxen baxen closed this Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant