Skip to content

Tests: shared output names come from core constants - #102

Open
TorecLuik wants to merge 35 commits into
test-suitefrom
use-core-output-constants-tests
Open

TorecLuik wants to merge 35 commits into
test-suitefrom
use-core-output-constants-tests

Conversation

@TorecLuik

@TorecLuik TorecLuik commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Refs #100 (harness only).

Contract test tests/test_script_output_constants.py: fails if a script in __workflows/ or _data/ sets or reads Message, URL, File_Annotation or Canonical_Inputs with a string literal instead of biomero.constants.results.*_OUTPUT, or defines its own output name. It reads the source only (no core needed) and skips for source revisions that do not use the constants yet; it includes a self-test of the detector. Existing tests that run script functions in a bare namespace now get constants.results from tests/script_output_names.py; the Canonical_Inputs source check accepts the old and new form.

Verified: against the new scripts (PR #101) 493 passed, 8 skipped; against master the new test skips and the touched tests pass; re-introducing a literal in a script fails the contract test.

Merge before #101. Remove the old forms after #101 merges.

Merge order (production and test PRs, interleaved)

Each step keeps master CI green: every (production state, test-suite state) pair on this path was run locally with core 2.9.0b26 from PyPI (482, 484, 484, 513, 524, 588, 594, 597, 606 passed; no failures). The production PRs merge cleanly in this order: each branch already contains the conflict resolutions with the PRs before it.

  1. test PR Tests: install biomero core 2.9.0b26 in the harness #103 (test-suite: biomero>=2.9.0b26,<3 in tests/requirements.txt, so core-dependent tests run in CI)
  2. test PRs Tests: schema imports, metadata view imports and workflow color #89, Tests: detached resume, cancel and registration #91 and Tests: per-directory canonical sidecars and unwritable folders #99 (capability-gated; they pass for the current master and for the new source)
  3. production PR Remove the input marker from results after import on every path #97
  4. test PR Tests: remove the input marker from results after import #98 (fails on master before Remove the input marker from results after import on every path #97, by design)
  5. production PR Import Zarr contracts from biomero-schema and own the workflow color #88
  6. production PR Detached processing: own launcher registration, resume and cancel in the scripts #90
  7. production PR Migration script: move canonical sidecars via importer helpers #95
  8. test PR Tests: shared output names come from core constants #102 (this PR)
  9. production PR Use the core script-output constants #101

A production PR's own check stays red until the test PRs before it in this list are merged (CI runs the shared harness from test-suite, which is not pinned); merge the test PR, then rerun the check.

🤖 Generated with Claude Code

Review fixes and chain

TorecLuik and others added 17 commits November 12, 2026 14:28
…e runner

Capability-gated: they follow the source revision that imports
biomero_schema.zarr and biomero.metadata_views, and skip or fall back for
older revisions.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Capability-gated: skipped until the source owns decide_workflow_resume and
core has WorkflowTracker.add_launcher_task. Existing structural tests accept
both the old and the new layout.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…gates

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…I, poll budget

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…tes, cancel script on a real tracker

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…pletes; scancel failure message

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Stubs accept the processed_folder keyword; new tests are skipped until the
matching source is available.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…er, other)

Also keeps the structural cancel-handler test, with a comment on why.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ants

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Contract test that fails if a script uses an output-name literal instead of
biomero.constants.results; existing tests that run script functions in a bare
namespace provide the constants. Gated on the source using the constants.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TorecLuik and others added 11 commits November 13, 2026 16:08
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ants-tests

test_ssh_recovery.py: #91 passes core's real constants to the pipeline
namespace, so the bare-namespace output names of this PR are not needed there.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- test_shared_output_names.py (renamed from test_script_output_constants.py)
  flags any string constant equal to one of the four shared names, except
  docstrings, the "type": "URL" value and the batched runner's UI_messages
  keys; fails when no scripts are found; only the per-script check is gated.
- script_output_names.py reads the values from biomero.constants.results.
- test_input_marker_cleanup.py gets the shared names from the same source.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The input marker cleanup (#97) merges before the Zarr contract move (#88) and
already imports its marker name from biomero_schema.zarr next to the
remaining biomero.zarr_contracts import. Count a script with any old-form
import as the old form, so the harness passes for source with and without
that cleanup.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
With core installed in CI, they failed on a missing file for the source
revisions that predate the detached ownership change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…nstants-tests

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…-tests

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Other core constants (such as the results path output) are already read by
earlier scripts changes; only the four shared output names mark a revision
that has moved to the constants.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@TorecLuik

Copy link
Copy Markdown
Collaborator Author

Independent review

Verdict: pass. Full review of its own commits (13b53e3, 0356fa0, 99f13b1 and the merge conflict resolutions); merged #89, #91, #99 and #98 content is reviewed in those PRs. Paired with production PR #101 (9def6e0).

Severity Where Finding
should fix (small) tests/test_shared_output_names.py:22-27 RESULTS_PATH_OUTPUT is in script_output_names.py but not in SHARED_OUTPUTS, so a literal 'Results_Path' in a script passes this test; only #98's text match (double quotes only) catches it. Mutation: 'Results_Path' not in import_outputs in the runner on #101 left all of #102's tests green. Add it to the map.
nit tests/test_shared_output_names.py:18-20 Only __workflows/ and _data/ are scanned. admin/ scripts still use "Message" literals (for example admin/SLURM_Cancel_Workflow.py:112); they have no reader, so this is fine, but say so in the docstring.
nit tests/test_shared_output_names.py:116-120 The gate also skips if every script reverts to literals, which looks the same as master. Acceptable and marked for removal after #101; do remove it then.

Runs (core 2.9.0b26, one run at a time): vs #101 606 passed / 9 skipped; vs #97 516 passed / 44 skipped; vs master 487 passed / 44 skipped / 29 failed, all 29 in test_input_marker_cleanup.py (the #98 tests, failing by design until #97 merges; #102 merges after #97 per the order). The 9 skips against #101 are environmental (omero missing, POSIX dir sync, symlinks, a core-version note). Running several full suites in parallel on one Windows machine stalled; one at a time they finish in about a minute.

Mutations: a literal 'URL' in _data/SLURM_Get_Update.py on #101: red (names file and line); the detector self-tests never skip. Mutation checks for the merged PRs were run with this tree against mutated #101 source (see the #91 and #99 reviews).

Nothing found for: values duplicated from core (names come from biomero.constants.results at runtime), silent skips of the self-tests.

🤖 Generated with Claude Code

TorecLuik and others added 7 commits November 16, 2026 14:15
Skip only when the source has no detached batched launcher (or no cancel
script); a source with launcher tasks but no register_detached_batches, or a
cancel script without runScript, now fails. Note where the processor key pin
is updated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The four tests skipped when a log message was reworded. Gate on the
processed_folder keyword / move_canonical_marker call the feature adds so a
reworded message fails. Drop a redundant guard assignment and an unused
tmp_path parameter.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@TorecLuik

Copy link
Copy Markdown
Collaborator Author

Resolved in 6acf0e4: RESULTS_PATH_OUTPUT is in SHARED_OUTPUTS; a 'Results_Path' literal in SLURM_Run_Workflow.py of #101 now fails the test. Merged the updated #91, #99 and test-suite so this branch carries their fixes.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant