Repository navigation
Normalize and consolidate shared LSP calls - #111
Conversation
|
Exact-head verification is complete at |
Keep the fix for #108 and drop machinery for states the daemon cannot reach: - Start the LSP in its own session and retire the whole process group through _kill_subprocesses. Cleanup retries until the group is verified gone instead of raising, so the runtime cannot release its lifetime lock or a cache slot while Lean workers survive. - Poison the session before a failed operation releases admission, so a queued call never reads the leftovers of a desynchronized stream. - A didClose failure or an exhausted budget poisons the session but keeps the result, preserving the existing response contract. - The runtime invalidates the cached session before aborting it on any non-busy failure. Removed as unreachable: async-cancellation bookkeeping (the daemon's signal handlers do not raise), LspCleanupError and its retry constants, and the ProjectResourceCache/services close changes, which the retry-until-clean abort makes redundant.
|
Golfed and re-verified at What changed and why
Diff vs base: 4 files, +1600/-90 → 4 files, +518/-27. Checks: exact-head CI green (test 3.10/3.13, real Lean, Windows hardening, CLA): https://github.com/facebookresearch/autoform-bot/actions/runs/37386310084 and https://github.com/facebookresearch/autoform-bot/actions/runs/37386314139. Locally: ruff clean; Remaining risks
|
hover() read the file inside the poisoning try, so a .lean file that is not valid UTF-8 poisoned the session, and the runtime's widened `except Exception` then invalidated it and killed its process group, failing any caller queued behind it. Read the file before admission, and let the runtime re-raise ValueError (which includes UnicodeDecodeError) without retiring the session. Stream decode errors are still wrapped as LspProtocolError, so this route cannot spare a broken stream.
|
Exact-head lifecycle review is complete at |
|
Independent review of Verdict: fix first (small). The Latin-1 hover regression is fixed and pinned. Moving the read ahead of admission opened a narrower one: an unreadable hover file now kills the session while another caller's request is in flight (issue 1). A two-line change closes it. Everything else below is low severity, golf, or a design question for a maintainer. Earlier findings
Confirmed issues1.
2.
3.
4.
Golf (about 30 lines)
Open design questions (unchanged since
|
|
Implemented the exact-head “fix first (small)” follow-up and merged current Production changes:
New regressions cover PermissionError during concurrent diagnostics, hover queue/budget exhaustion, both operations staying healthy on admission failure, a 5,000-digit JSON integer frame, and faithful poisoning in the cleanup-order test. Validation on the merged head:
One separate merge blocker remains. The generic That prerequisite belongs under #109 as a focused cache-retirement change: keep a per-root retiring reservation through verified cleanup, count it against capacity, and make same-root acquisition wait/fail rather than invoke the factory. Until it lands, #111 should remain draft and should not close #108. The small fixes above are orthogonal and can stay on this branch. |
|
Restacked at |
Review at 2ce7ecbVerdict: merge-ready once #168 lands. Against its base (#168 at Nits
Earlier reviewsStatus at this head of the findings from earlier swarm reviews: the posted 10-06 review (58ff1a2) and earlier unposted notes at ee352e5. After the restack, most of that code lives in #168, which is reviewed separately.
Landing
|
json.loads raises RecursionError, not ValueError, when a body nests arrays or objects past the decoder's recursion limit. A 400 KB body of 200,000 nested arrays, far under MAX_LSP_MESSAGE_BYTES, therefore reached callers as a raw RecursionError instead of LspProtocolError. The session was already retired correctly; only the reported error was wrong. Catch RecursionError alongside ValueError and pin it with that body, which exceeds the limit on both Python 3.10 and 3.13.
|
Fixed the review nit in |
The graceful close test first asserted get_diagnostics() == []. Its quiet-period reads give up on a frame whose header is still arriving when the one-second window ends, a pre-existing behaviour main shares, and one real-lean run on #111 failed there. The test is about close(), so open the file with didOpen and wait for the file worker, as the abort test already does; both tests now share that helper.
|
Merge status at
|
Deicyde
left a comment
There was a problem hiding this comment.
Status at fd9197f2: blocked on #168, otherwise ready
- Contents: #111's own change on top of #168 is +133/−58: the
_admittedand_lsp_operationfolds, plus the JSON body decode hardening. It also carries #168's final-review fixes, brought in by the clean mergefd9197f2. - Review: the swarm reviewed it at
2ce7ecb4("merge-ready once #168 lands") and found one nit, fixed in0f789c28. Since then the only changes are the two merges of #168's fixes (869e0afd,fd9197f2), both without conflicts. - CI: all 9 check runs at this commit pass, including real Lean. The earlier real-Lean failure on
869e0afdwas a flake in a test that no longer callsget_diagnostics; the root cause is #238. - Mergeable: GitHub reports CLEAN against the base
fix/retained-project-cleanup. A simulation of the full order (merge #168 into currentmain2825e93b, then this PR) also produces no conflicts. - To merge: once #168 lands, retarget this PR to
main. Check that the diff shows only #111's own +133/−58, rerun CI, then squash-merge. Squash matters because the restack merge8d74b484brings superseded pre-restack history in as its second parent. - Downstream #105: GitHub now marks #105 as conflicting with this branch. The only conflict is in
tests/test_shared_lean_runtime.py, where #105 replacestest_partial_repl_startup_transfers_cleanup_without_masking_the_error. Take #105's side when you next update it from this branch.
Brings in #172 (implementation notes), #111, #168 and #223 to #229. Three files conflicted, each because both sides added text at the same spot; every resolution keeps both: - AGENTS.md: main's rewritten roadmap bullet (implementation notes under blueprint/.implementation-notes/), then this branch's import bullet. - skills/formalize/SKILL.md: this branch's "renew the complete claim set" joined with main's audit run and stale-note deletion, rewrapped. - tests/test_skill_examples.py: this branch's import-scope test and main's implementation-notes test, side by side.
Merge main at 82cca35, which carries the final #168 (b553a92) and #111 (521a940) plus #172, #175, #186, #188, #198, #208, #209 and the audit PRs #223 to #229. Main's changes are kept as they are, and this branch's changes apply on top of them unchanged. Only tests/test_shared_lean_runtime.py conflicted, in two places. The imports keep both sides: main's servers.lsp.server import, which #168's new live-process LSP timeout test needs, and this branch's servers.repl.core header-command import. The second conflict is in test_partial_repl_startup_transfers_cleanup_without_masking_the_error, where #168 relaxed a timing bound from 0.5 to 1.5 seconds. This branch replaces that test with test_dirty_repl_generation_transfers_to_retirement_before_lease_release because a cold pool no longer starts a REPL while it is built, so the replacement stays and #168's edit leaves with the old test. The other files both sides changed merged without conflict: servers/lean_runtime.py, the CI workflow, tests/test_repl_core_protocol.py, tests/test_skill_examples.py, skills/setup/SKILL.md and the example README.
Brings in main at 82cca35: the Lean client's request phases (#175), the skeleton probe's roots passed as JSON (#209), autoform unused-hypotheses (#198), the shared checkout pin check (#186), the tree snapshot scan closure (#188), the statement contract follow-ups (#208), retained project cleanup and LSP process ownership (#168, #111), implementation notes outside articles (#172), and the audit fixes #223 to #229. Conflicts: - autoform_cli/audit.py: this branch's validate_coverage_roles import beside main's IMPLEMENTATION_NOTES_DIR import. - autoform_cli/graph.py: this branch's ATLAS_SCHEMA and _MAX_ATLAS_BYTES beside main's IMPLEMENTATION_NOTES_DIR. - autoform_cli/README.md: this branch's area, atlas and catalog paragraphs, then main's implementation-notes paragraph. The publication contract keeps this branch's explorer description and takes main's "Hidden files, including .implementation-notes, are omitted". - skills/roadmap/SKILL.md: main's implementation-notes paragraph after the article-ids instruction, then this branch's paragraph break before the claim rule. Main's "Assert formalization or mathlib: true only after exact verification" stays out there, because this branch moved that rule into its own assertion paragraph further down. - tests/test_visualization.py: both new tests, this branch's checked catalog labels and main's structure page without implementation notes. One change had no textual conflict. #172 has every command that loads the graph reject malformed, empty, symlinked or orphaned implementation notes. Render loads the graph from its capture, which leaves hidden files out, so it skipped that check. It now checks the notes in the vault, the same fix #165 makes for its own capture. The notes are never published or hashed, so reading them in place changes no output. A new test renders a vault with an orphaned note and expects render to stop before it writes anything.
Small follow-up to #168. That PR now owns verified process-group cleanup, retained cache retirement, startup handoff, and failed-session fencing; this restack removes the superseded implementation from #111.
Changes
_admittedwrapper for diagnostics and hover. It preserves the established ordering: hover input is read before admission, queue/deadline failures do not poison a healthy session, and an admitted failure marks retirement before releasing the operation lock._lsp_operationruntime wrapper for both public LSP methods. It preserves Retain Lean project ownership through verified cleanup #168's canonicallease_resolved/invalidate_resolvedkeys, state-based error routing, successful-but-poisoned retirement, and asynchronous cleanup ownership.LspProtocolError. CPython raises a plainValueError(notJSONDecodeError) when an integer exceeds its configured digit limit, andRecursionErrorwhen arrays or objects nest past its recursion limit.Regression coverage
didClosebudget returns the completed result but retires the session;Scope
No process, cache, deadline, or public MCP contract is introduced here. #168 owns those contracts. This PR is a +133/−58 consolidation and normalization follow-up.
Validation
At exact head
2ce7ecb4over #168 head953ff2f7:git diff --check: pass;make check-example: pass, including strict MkDocs;0f789c28then adds theRecursionErrorcase. On it,tests/test_lsp_protocol.pypasses (31 tests) on Python 3.13, the new test fails with a rawRecursionErrorwithout the fix, and Ruff andgit diff --checkpass. The full-suite and CI results above predate that commit.