TASK-045: headless periodic checkpointing (pyflow record) - #77
Merged
Conversation
Opens Stage 8 (Recording & Playback) with its first task: a new pyflow record subcommand that steps a simulation forward with no rendering window at all, writing self-contained, resumable checkpoint files at a bounded, config-driven interval. bootstrap.py's simulation- state construction/advancement logic is extracted into simulation_run.py so the headless path can reuse it without paying for a RenderWindow; checkpoint.py holds the torch.save-based file format and the restore_simulation_state logic needed to resume a "passive"-mode run correctly. Deterministic resume-from-checkpoint is verified bit- identical and confirmed to have real teeth by deliberate mutation. Replay and playback (TASK-046/047) are deliberately out of scope, named as such throughout the stage's own Completion Criteria and Discharge map, along with the two backlog/doc obligations they leave open (docs/planning/backlog.md, docs/architecture/sequences.md). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
README's Current Phase still said Stage 8 was "not yet started" after TASK-045 landed -- the fourth time this exact section has gone stale this way, and its own prose already documents the first three. Fixed, and added the user-facing walkthrough that was missing from the original PR: a verified `pyflow record` example against Heat Diffusion plus a real checkpoint inspection snippet, both run for real and their output pasted in rather than guessed. Also found while checking: Stage 8's own "Status as of" heading used free text that satisfied check_stages.py's loose prefix match but not generate_status_report.py's stricter template, so the status line was silently invisible to the checker meant to catch exactly this kind of staleness (complete_claimed parsed as None, not False). Rewritten to the template; verified directly against parse_roadmap rather than only against make check-status passing. Added checkpoints/ to .gitignore -- RecordingConfig's own default output_dir, now that README documents running it from the repo root. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A user asked whether resuming from a checkpoint is really deterministic given the prescribed velocity field isn't checkpointed. It is -- the field is a pure function of the checkpoint's own embedded config, not evolved state -- but tests/unit/test_recording_determinism.py's only fixture had zero prescribed velocity, so a reconstruction bug that silently defaulted to zero would have passed unnoticed. Added a nonzero- velocity fixture and two tests pinning this: one at the trajectory level, one a direct ground-truth check of the reconstructed field. Both were run under two deliberate mutations to confirm they actually catch what they claim to: a corrupted restore_simulation_state (caught by the trajectory test, missed by the old zero-velocity one -- proving the gap was real) and a config parser silently dropping the velocity field (missed by an earlier differential draft of the direct test, which compared two calls sharing the same buggy code path, until it was rewritten to check against a hand-computed expected value instead). Also adds `pyflow resume --checkpoint <file> --max-frames N`: continues a headless recording from an existing checkpoint, no --config at all (the checkpoint carries its own), sharing its checkpoint-writing policy with `record` through a new _advance_and_checkpoint helper -- confirmed to genuinely share behaviour, not just source, by an off-by-one mutation that broke both functions' own tests together. Still recording's own scope, not the deterministic windowed replay or playback TASK-046/047 still own -- no rendering, no dense per-frame materialization. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Opens Stage 8 (Recording & Playback) with its first task: a new
pyflow recordsubcommand that steps a simulation forward with no rendering window at all, writing self-contained, resumable checkpoint files at a bounded, config-driven interval.simulation_run.pyextractsbootstrap.py's simulation-state construction/advancement logic so a genuinely headless path can reuse it without ever constructing aRenderWindow(verified behaviour-preserving by the full pre-existing suite passing unmodified).checkpoint.pyholds thetorch.save-based,weights_only=True-loadable checkpoint format (config embedded asdataclasses.asdict, no per-field type tag needed sincePressureFieldis never checkpointed) andrestore_simulation_state, which reconstructs a resumableSimulationStatecorrectly for "passive" mode (whose prescribed velocity field is never itself checkpointed).recording.pynever importsrendering/pygfx/rendercanvasat all -- headless is structural, not a default.rtol=0, atol=0) and confirmed to have real teeth by deliberately corruptingrestore_simulation_stateand watching the test fail before trusting it green.Replay and playback (TASK-046/047) are deliberately out of scope, named explicitly throughout Stage 8's own Completion Criteria and Discharge map, along with the two backlog/doc obligations they leave open (
docs/planning/backlog.md,docs/architecture/sequences.md's Section 3).Test plan
make cigreen in full (lint, mypy --strict, full suite, all doc-consistency checks)restore_simulation_state, confirmed the test failed, reverted, confirmed green)pyflow record --config examples/golden-demos/heat_diffusion.yaml --max-frames 200 --output-dir ...produced the expected 3 checkpoint files; inspected one directly withtorch.loadto confirm shape/contenttests/integration/test_record_cli.pyrunspyflow recordas a real subprocess against a real golden demo🤖 Generated with Claude Code