Skip to content

Add and activate disposable REPL execution - #43

Closed
Deicyde wants to merge 5 commits into
facebookresearch:mainfrom
VivienCabannes:split/01b-repl-runtime-safety
Closed

Deicyde wants to merge 5 commits into
facebookresearch:mainfrom
VivienCabannes:split/01b-repl-runtime-safety

Conversation

@Deicyde

@Deicyde Deicyde commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Stack position: 2 of 3. Builds on #13 (merged as afaf215). The generation-isolation slice follows as a separate PR once this lands.

This PR introduces LeanRepl.run_disposable() and routes every public pooled REPL call through it. Each call gets a fresh child, exactly one request frame, strict response validation, and verified process-group cleanup before a result is returned.

The cutover also:

  • carries one absolute operation deadline across daemon startup, project admission, header validation, worker startup, and execution, with separate bounded cleanup grace;
  • validates strict private request and response envelopes;
  • never replays a request after dispatch may have occurred;
  • returns typed pre-dispatch errors and explicit outcome-unknown failures;
  • removes process-owned env and proofState handles;
  • bounds REPL responses and header-parser stdout/stderr;
  • quarantines a project pool when child cleanup cannot be verified, while keeping pool-shutdown deadlines aggregate;
  • validates the original submitted header with the active toolchain's lean --deps-json, before adding warmup imports, and accepts the known Lean 4.19 and current output schemas strictly;
  • resolves the parser through Lake's LEAN_SYSROOT, so a project executable cannot shadow lean;
  • invokes the package-qualified @repl/repl target and documents the required immutable consumer dependency;
  • pins the tested upstream REPL revision for Lean 4.32.2 in the setup example and real-REPL fixture;
  • exercises consecutive isolated calls through the public client, daemon, pool, and REPL path in CI.

Import filtering limits calls to known libraries; it is not a security sandbox, because submitted Lean can still run arbitrary IO.

Validation: Ruff, the full Python suite, the setup example check/render, both pinned REPL builds, and 11 real-REPL tests pass locally. The cleanup-grace fix from #13 is preserved.

A disposable call sends its whole header to Lean with no environment,
but the allowlist read it line by line. Lean skips comments before and
between imports and accepts several imports on one line, so a header
such as a block comment followed by an import, or two imports on one
line, reached Lean without its later modules being checked.

The disposable path now scans the header the way Lean does, reading
every import across comments and lines, and rejects header syntax the
scanner does not model (module, prelude, public/meta/private import,
import all) so validation fails closed.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 26, 2026

@Deicyde Deicyde left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please replace the hand-written header scanner (_header_import_modules, servers/repl/core.py:298) with Lean's own header parser, via lean --deps.

Right now the allowlist check depends on a Python reimplementation of Lean's header grammar. It matches Lean on every case tried so far, but any gap between the two is an allowlist bypass, and the grammar changes across Lean versions (module, public/meta import, import all are recent). Lean already ships the parser we need:

  • lean --deps file.lean parses the header with Lean's real parser and prints the resolved file for each direct import. Run it under the worker's lake env so it resolves against the same search path as the REPL.
  • On Lean 4.32.0 it handled all the tricky headers correctly: comments before and between imports, several imports on one line, comments inside a module name, module/public import/import all, prelude, and an import after the body.
  • An unresolvable module makes it exit nonzero, so failure rejects by default.
  • Each call took well under a second, which is small next to the fresh Mathlib-importing REPL each disposable run already starts.

Suggested shape:

  1. Write the submitted code to a temp file and run lake env lean --deps <file> with the remaining deadline.
  2. Nonzero exit or timeout: return Rejected Lean header: ... with Lean's stderr.
  3. Map each printed path back to a module name by stripping the LEAN_PATH entry it sits under and the .olean suffix, then check the root against allowed_imports as today. Skip the implicit Init entries.
  4. Delete _header_import_modules and the special-case rejections for module/prelude/modifiers, since Lean now handles those.
  5. Keep the existing adversarial header cases as tests against the new path. They make good regression tests for the --deps integration.

Optional, same PR or a follow-up: the stateful run() path still uses the line-based _split_imports_and_body for its allowlist check (core.py:400). It isn't exploitable today, because stripped-header code reaching Lean as body can't import anything, but using one parser for both paths would remove the second copy of the header rules.

A compiled Lean helper built on Lean.Parser.parseHeader + Elab.headerToImports also works and returns module names directly. --deps is preferred because it needs no extra build artifact tied to the toolchain.

Replace the hand-written Python header scanner with `lean --stdin --deps`,
run under `lake env` so it resolves imports against the worker's search
path. Each resolved .olean path is mapped back to its module name and its
root checked against the allowlist. A header Lean rejects, an unresolvable
import, a path outside the search path, or a timeout rejects the run.

The check now runs on the exact text sent to Lean, after warmup imports are
prepended, and no longer needs special cases for `module`, `prelude`, or
import modifiers. Adversarial header cases move to the real-REPL job so they
run against the pinned Lean.
…ines exact

`lean --deps` reports resolved .olean paths, and mapping them back to module
names misreads a quoted atomic name such as «Mathlib.X» as root Mathlib.
`lean --deps-json` reports each module name as Lean spells it, needs no
search-path mapping, and reports header parse errors in its output, which
now reject the run. Its output shape is unchanged from Lean 4.26 to 4.34.

LeanRepl.close_with_deadline no longer widens the pool's shared deadline per
worker, so shutting down several workers cannot outlast the cleanup budget.
close() still reserves cleanup time after an expired request deadline.

Also remove the unused startup_stagger setting, document that import
filtering is not a security sandbox, and add a real-REPL test that the same
declaration succeeds on two consecutive pool calls.
@Deicyde

Deicyde commented Sep 26, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by the direct Lean Beam persistent-session architecture in draft #46. That replacement removes the custom REPL/runtime stack instead of extending disposable child execution. The branch remains available for history; #46 is explicitly blocked from merge until its upstream Beam release gates are satisfied.

@Deicyde

Deicyde commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Restored for active work as draft #101 on a facebookresearch-owned branch. The replacement is merged with current main and preserves the original REPL safety design without resuming development on the old fork branch.

@Deicyde

Deicyde commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Update: draft #101 was decomposed and closed. Active replacements are #105 (minimal disposable lifecycle), #106 (Lean-native header validation and pinned dependency), and #107 (fault injection and real-REPL CI).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant