Skip to content

fix: recover Claude native delivery beyond the transcript tail - #1647

Closed
Masashi-Ono0611 wants to merge 7 commits into
fujibee:integration/agmsgd-betafrom
Masashi-Ono0611:codex/claude-transcript-observation
Closed

Masashi-Ono0611 wants to merge 7 commits into
fujibee:integration/agmsgd-betafrom
Masashi-Ono0611:codex/claude-transcript-observation

Conversation

@Masashi-Ono0611

@Masashi-Ono0611 Masashi-Ono0611 commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

When a delivered Claude notice falls outside the last 512 KiB of a growing transcript, the current observer can leave that notice pending indefinitely and prevent the next notification. Replace the tail-only lookup with a bounded streaming observer that retains complete-row checkpoints, cold-recovers persisted pending notices, and confirms only complete delivery evidence.

Verify the candidate row against the same file before confirmation; invalidate checkpoints after detected rewrites while allowing normal appends. The daemon yields between at most 64 KiB scan/proof units and cancels observation on stop, closing its descriptor and leaving unconfirmed delivery pending. The synchronous helper remains available for existing callers.

This Draft targets integration/agmsgd-beta: the affected daemon channel is present in the v1.6 RC branch and absent from current main. It changes two internal modules and their regression tests, without a storage migration, new dependency or public command. No release/version bump is proposed.

Refs #1644.

Test Coverage

transcript -> bounded parser -> complete-row checkpoint -> candidate reread proof
                        | held/unreadable -> pending, no duplicate send
                        | delivered -> confirm -> next notification
async driver -> yield between units -> abort on stop -> close descriptor

Independent contract audit: 26/30 contracts covered (86.67% value-weighted), zero weak paths, four explicit gaps. Proposed follow-ups cover invalid/missing-file admission, rotation/shrink/zero-read during proof, terminal queue cleanup branches, and stop-during-send/default-instance isolation. The changed test file grows from 5 to31 top-level tests (+26); this audit did not author or reject tests. No independently inspected base-green/red-proof counter is claimed.

Final Node24 run:49 passed across Claude queue, Codex queue and daemon main; two Bats wrappers passed. The evidence ledger records exit0 but does not parse these runners' counts; counts above come from their actual output. Mechanical preflight passed. The automatic lane selector also ran37 Node tests successfully and retained two uncleared remote obligations; they are not represented as passes.

The original bats tests/ run finished exit1:2859 cases reported,2707 passed,133 failed,19 skipped. All133 failures also reproduced on clean RC:131 terminal-detection entries resolve with child-only desktop-marker removal; two SQLite compatibility failures remain. See #1646 and #1645. The run began before the focused fixes and was not frozen on this final commit throughout; the final49+2 affected checks above are separate. Full-suite green and a frozen-final-head whole-suite pass are not claimed.

Pre-Landing Review

Current native, parent and two outside passes found no P0–P2 finding on the serialized daemon path. Review binding matches the frozen content; the review dashboard remains UNVERIFIED because informational findings remain. Outside passes completed with clean/P3 verdicts, not runtime certification.

Gate edits: none detected (4 listed,3 read,0 findings).

Retained limitations: cold recovery still consumes total CPU/I/O proportional to file size and delays later seats; filesystem calls within a work unit are synchronous. Arbitrary same-inode rewrite-plus-growth behind a checkpoint and direct overlapping same-key helper calls remain conditional investigations; no overlapping production caller was found. Metadata-only changes can conservatively trigger a rescan. Giant unterminated rows require reparse and delivery rows require a newline. Dense near-empty rows are a performance follow-up, without an observed control timeout.

Exploratory QA

Ten selected scheduling, cancellation, default-channel recovery and rewrite/append probes passed. On one synthetic64 MiB Air run, the prior synchronous scan delayed the first scheduled heartbeat about6.8s; the cooperative implementation allowed it at7.34ms while total scan time remained about7.9s. This is a single synthetic observation, not a latency SLA or a reduction in total work.

Four fresh native request/receipt/result round trips completed with this observer in an isolated local pilot, with retained Claude conversation context and automatic Codex owner collection. The first two covered restart/next-notice acceptance; the later two were useful operations-document review and verification of its corrections. That dogfooding identified two guard gaps in the separate local harness (runtime pin verification and saved review inputs), which were repaired there and verified with23 harness tests; those changes are not bundled into this three-file observer PR. Controlled UI stop/start was observed; the dedicated pilot remains running for continued local use. These are local runtime evidence, separate from mocks and CI; no private transcripts or identifiers are included here. App-wide cold restart, archive-only zero-input recovery, sleep/logout/network recovery and native Codex project-hook activation remain unverified.

Design Review

No frontend files changed — design review skipped.

Eval Results

No prompt-related files changed — evals skipped.

Scope Drift

Scope Check: CLEAN. The delta addresses the native transcript observation failure, its rewrite safety and cooperative stop behavior. Unrelated terminal-fixture and SQLite compatibility defects are recorded separately, with no fixes bundled into this branch.

Plan Completion

Plan completion audit: not run (no plan is bound to this branch and no docs/designs/ file matches). Fix: add "Plan: " to the PR body, or run /autoplan.

No plan-specific verification items were selected. Diff-scoped functional QA was executed separately. The personal Air environment plan is not this OSS branch's plan.

Verification Results

Human-authorized Ready-for-review exception (2026-10-17): the user explicitly authorized converting this Draft to Ready for review while retaining the observed failed full-suite verdict and the separately reported issues #1645/#1646. This accepts the absence of a frozen-final-head whole-suite pass. The installed ship-measure helper is unavailable, so its prescribed repeated-shard diagnostic has not run; baseline/control diagnostics are not substituted for that gate. The automatic lane selector retains uncleared Linux/server-check and Windows/app-test obligations. Local runs do not clear those records, and a skipped CI job would not prove that platform's observer behavior. No merge, release or deployment is authorized.

The mechanical ship-receipt attempt refused PREGATE_REMOTE_UNCLEARED and wrote nothing. Its gate records were preserved. The PR was initially opened as a Draft under the prior human exception; the subsequent Ready-for-review authorization does not clear any gate or imply a fabricated green receipt. CI results are reported below.

CI results (completed)

At the frozen PR head, verify-versions succeeded. The Tests workflow completed with failure: 24 jobs succeeded and two failed (Ubuntu shard5 and the aggregate bats result). All five macOS Bats shards passed; Ubuntu shards1–4 passed; Windows-specific Bats jobs passed. Ubuntu shard2 and macOS shard2 each ran the Claude Code queue support Bats test without a skip marker.

Ubuntu shard5 failed delivery stop: remains global — kills watchers across all projects: wait_for_pid_exit "$pid_a" timed out, with trap: line 2: unexpected EOF while looking for matching ')'. This matches the previously reported signature in #1435. The new occurrence comment records the current run and bounded diagnostics. Candidate/base selected-case diagnostics passed once locally; this neither clears the CI failure nor establishes its cause. No CI rerun or test relaxation was performed.

This workflow does not run node --test for the changed observer unit-test files. The focused Node24 run (49 passed) and two Bats wrapper tests are local evidence; only the wrapper Bats coverage above is demonstrated by CI. The green Windows app-test and remote-server job statuses are no-change skips and do not clear the two automatic remote obligations. The CI result is not all-green.

The original local bats tests/ result remains exit1 (2707 passed, 133 failed, 19 skipped of 2859, on mixed source input); all133 also reproduced on clean RC. No frozen-final-head whole-suite pass is claimed. Human-approved scope is Ready for review with the known failed tests and uncleared gates disclosed; no merge, release or deployment is authorized.

Documentation

Status: current — no documentation correction is required by this candidate.

Scope: read-only static re-audit of base 3fb012a through head b8da8bd on codex/claude-transcript-observation. The candidate has six commits and three changed files, no staged, unstaged or untracked changes, and all recorded hashes match. Recursively inventoried tracked/nonignored authored docs; revalidated the fifteen previously full-read relevant entry points, bilingual architecture/contributor docs, nested design docs and authored skill templates by matching their current hashes against the earlier read version. Reviewed both subsequent source/test deltas and the complete base-to-candidate scope. No images or private/runtime transcripts were read. This is documentation coverage, not live-delivery verification.

Documentation health:

  • README.md — Current for this change; native Claude delivery, conservative unreadable evidence handling, enable/status, stop and fallback instructions still apply.
  • README.ja.md — Current for this change; pre-existing absence of the English beta section remains translation debt.
  • ARCHITECTURE.md — Current for this change; no driver, storage or dependency contract changes.
  • ARCHITECTURE.ja.md — Current for this change; pre-existing three-axis/four-axis translation difference remains debt.
  • CONTRIBUTING.md — Current for this change; existing small-bug-fix policy applies.
  • CONTRIBUTING.ja.md — Current for this change; contributor flow is unchanged.
  • SKILL.md — Current for this change; generated native-delivery branch remains consistent with authored overlay.
  • llms.txt — Current for this change; stable external script surface is unchanged.
  • scripts/skill-base.md — Current; no new user command, option or configuration is needed.
  • scripts/drivers/types/claude-code/template.md — Current; delivery=native continues to bypass Monitor setup.
  • docs/design.md — Current for this change; existing hook design and public script responsibilities are unaffected.
  • docs/design.ja.md — Current for this change; same boundary as English design.
  • docs/design/agmsgd-architecture.md — Current for this change; future 2.0 design is unaffected; its pre-existing beta scope note lags README.
  • docs/design/agmsgd-architecture.ja.md — Current for this change; future design is unaffected; pre-existing beta translation gap remains.
  • docs/building-on-agmsg.md — Current; external integrations still use the documented scripts rather than internal daemon modules.
  • AGENTS.md — Skipped; absent from this checkout.

Coverage map:

Changed surface Reference How-to Tutorial Explanation
Claude native notice delivery README beta section Existing enable/status/fallback instructions Existing first-run flow Existing delivery boundary
Internal createTranscriptObserver/createAsyncTranscriptObserver and lifecycle cache Source comments and regression fixtures Internal only; no new user action Not applicable Shared generator, bounded cooperative scan/proof units, complete-row checkpoints, cold recovery, rewrite invalidation and candidate-row verification are explained in source
Channel stop cancels active observation README stop/recovery instructions and source contract Existing daemon stop/start workflow Not applicable Source and regression fixtures explain descriptor closure and retention of unconfirmed pending notices
Unreadable/rewrite evidence stays pending README conservative handling and unchanged resolvePendingDelivery contract Existing status/fallback path Not applicable Source comments and regression fixtures; cancellation also stays pending

Documentation debt: no candidate-induced zero-coverage public surface or reference-only public feature. Existing debt predates the supplied base: beta translation/scope synchronization, three-axis/four-axis wording, the stale Codex monitor restriction in README sandbox prose, and discoverability of nested agmsgd design pages. These are separate reference/explanation maintenance tasks; this audit does not approve a rewrite or expand the candidate.

Diagram drift: no entity was renamed, removed, moved or split by this candidate. Existing README/design tree and Mermaid diagrams do not describe transcript observer/cache internals, so no diagram update is required.

Test plan

  • Node24 node --test tests/agmsgd_claude_code_queue.test.mjs tests/agmsgd_codex_queue.test.mjs tests/agmsgd_main.test.mjs:49 passed.
  • Child-only marker removal, bats tests/test_agmsgd_claude_code_queue.bats tests/test_agmsgd_codex_queue.bats:2 passed.
  • Ten selected functional QA probes; four native pilot round trips, including useful local dogfooding and its correction review, plus controlled lifecycle checks.
  • Whole-suite and remote validation are not all green; see Verification Results.

Generated with Codex; outside review used Claude. No merge or deployment is requested.

@fujibee

fujibee commented Oct 20, 2026

Copy link
Copy Markdown
Owner

@Masashi-Ono0611

Thank you for the detailed report in #1644 and for this PR, including the synthetic reproduction and the tradeoffs you wrote down. It pinpointed the cause.

I'm closing this in favor of a smaller change in #1662. It records the transcript length when a notice is sent and confirms delivery from that offset onward, keeping the existing delivered/held evidence rules. Keeping the observer small matters here because the same file is also changing to authenticate notices with the session's messaging token.

Your rewrite and restart scenarios are useful test cases, and I'll check them against the smaller fix. A re-test on rc.5 would be very welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants