Skip to content

Tests: remove the input marker from results after import - #98

Open
TorecLuik wants to merge 19 commits into
test-suitefrom
test/remove-input-marker-after-import
Open

TorecLuik wants to merge 19 commits into
test-suitefrom
test/remove-input-marker-after-import

Conversation

@TorecLuik

@TorecLuik TorecLuik commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Tests for #96 (production change: PR for branch fix/remove-input-marker-after-import on master; merge that one first, because these tests fail on current master by design).

tests/test_input_marker_cleanup.py:

  • One test per result path (remote shallow, importer removes the marker, no importer evaluation), each run with the importer removing the marker as today and with it no longer removing it. They drive the real importResultsToOmero with a fake import script that asserts the marker still exists when the importer runs, so cleanup before the import fails.
  • Helper tests: removes every store's marker, only the exact file name, never outside the result folder, never below a store root, repeatable, a removal error only warns.
  • Decision tests: skip at DEBUG when shallow Zarr is off, WARNING when no Results_Path came back, marker stays when the import fails.
  • Review follow-ups: the result folder and <result folder>/<wf> are refused, a result folder linked to user data is refused (skipped where links cannot be created), links are not followed, an unreadable folder warns, and a failing cleanup does not fail the import.
  • The result folder is fixed in 1.8 (configurability: Configurable result folder (.analyzed) for the importer route NL-BIOMERO#111): the sweep refuses a Results_Path whose parent folder is not this run's workflow id and accepts the path the import makes; only SLURM_Import_Results.py contains the literal .analyzed, exactly once (AST test); the retry lookup is tested.
  • Search depth: the marker sits at the root of each exported store, one level below the result folder; a marker deeper inside a store is not removed and the walk never descends into a store.
  • The test file uses TRANSFER_INPUT_MARKER of biomero_schema.zarr and core's constants instead of its own copies; the shared import environment is one helper and the result-path table has one row per distinct path. test_import_retry_storage.py gives its bare namespace read_analyzed_data_folder.
  • A contract test that both scripts agree on the Results_Path output.

Checked red/green on scratch copies: all fail on unchanged master; cleanup call removed, cleanup before the import, realpath to abspath, followlinks=True, a looser folder check, no try around the call, no onerror, and a second .analyzed literal, removing the workflow-id check and removing the depth limit each make at least one test fail. Full harness against #97 locally (core 2.9.0b26 from PyPI): 518 passed, 35 skipped; against master, only this file's tests fail.

Merge order (production and test PRs, interleaved)

Each step keeps master CI green: every (production state, test-suite state) pair on this path was run locally with core 2.9.0b26 from PyPI (482, 484, 484, 513, 524, 588, 594, 597, 606 passed; no failures). The production PRs merge cleanly in this order: each branch already contains the conflict resolutions with the PRs before it.

  1. test PR Tests: install biomero core 2.9.0b26 in the harness #103 (test-suite: biomero>=2.9.0b26,<3 in tests/requirements.txt, so core-dependent tests run in CI)
  2. test PRs Tests: schema imports, metadata view imports and workflow color #89, Tests: detached resume, cancel and registration #91 and Tests: per-directory canonical sidecars and unwritable folders #99 (capability-gated; they pass for the current master and for the new source)
  3. test PRs Tests: retry harness knows the result folder constant #104 and Tests: harness namespaces know core's Canonical_Inputs name and the shallow-storage flag #105 (test-suite: the retry-lookup tests know the RESULTS_FOLDER constant; the harness namespaces know core's Canonical_Inputs name and the shallow-storage flag; without them Remove the input marker from results after import on every path #97's CI fails with NameError)
  4. production PR Remove the input marker from results after import on every path #97
  5. test PR Tests: remove the input marker from results after import #98 (fails on master before Remove the input marker from results after import on every path #97, by design) (this PR)
  6. production PR Import Zarr contracts from biomero-schema and own the workflow color #88
  7. production PR Detached processing: own launcher registration, resume and cancel in the scripts #90
  8. production PR Migration script: move canonical sidecars via importer helpers #95
  9. test PR Tests: shared output names come from core constants #102
  10. production PR Use the core script-output constants #101

A production PR's own check stays red until the test PRs before it in this list are merged (CI runs the shared harness from test-suite, which is not pinned); merge the test PR, then rerun the check.

🤖 Generated with Claude Code

  • Second review round: the guarded biomero_schema.zarr import and its constant are pinned against the file name the tests use (the transfer-marker test uses the constant too), importer off / shallow off / both off and an empty Results_Path are covered, a ../ escape path is refused, fake outputs use an rstring-like object, and the textual contract is commented. test-suite is merged in.

  • Core: tests/requirements.txt needs biomero>=2.9.0b26,<3. The tests inject the constants for the run script; the contract test checks that both scripts use the core constant and define none locally.

TorecLuik and others added 5 commits November 13, 2026 14:02
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ants

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@TorecLuik

Copy link
Copy Markdown
Collaborator Author

Independent review

Verdict: pass. Full review, paired with production PR #97 (8d6ed48).

Severity Where Finding
nit tests/test_input_marker_cleanup.py:24-27 MARKER = ".biomero-input.json" duplicates the schema constant imported two lines above and pinned at line 469. Use TRANSFER_INPUT_MARKER directly.
nit tests/test_input_marker_cleanup.py:82-83 RESULTS_FOLDER_NAME and ZARR_ROOT_FILES are copies of the source's module constants; a changed source constant goes unnoticed. Read them from the parsed source.
nit tests/test_input_marker_cleanup.py:422-451 test_a_failing_cleanup_does_not_fail_the_import repeats the setup of fake_script_environment (362-376); give that helper a way to override a function.
nit tests/test_input_marker_cleanup.py:350-359 remote-kept-full and local-evaluation are the same tuple, so the same case runs twice.
nit tests/test_input_marker_cleanup.py:472-488 The contract test matches formatted source text; a reformat fails it without a behaviour change. #102's AST detector covers the literal side; consider matching on the AST.
note tests/test_input_marker_cleanup.py:230-256 The two symlink tests skip where links cannot be created (Windows without privilege; a junction fallback is tried). The call-recording test at 265 covers followlinks=False everywhere.

Runs: vs #97 511 passed / 9 skipped. Vs master the 29 tests of this file fail by design until #97 merges (no other file fails). Vs #101 needs #102 first.

Mutations on #97 source:

  • cleanup call removed: red (every-path test).
  • followlinks=True: red (test_the_search_never_follows_links).
  • no pruning at store roots: red (test_removes_only_files_named_like_the_marker).
  • realpath -> abspath: red (linked-to-user-data; ran here through a junction).
  • flag check if not SHALLOW_ZARR_STORAGE_ENABLED -> if False: red (3 params).
  • except Exception -> except KeyError around the cleanup call: red.
  • removed += 1 -> += 2: red.
  • double-quoted literal "Results_Path" in the runner: red (contract test); a single-quoted literal would pass it (see Tests: shared output names come from core constants #102 review).

Nothing found for: silent skips; flag states (shallow off, importer off, both off, empty and missing output) are covered with exact log levels.

🤖 Generated with Claude Code

TorecLuik and others added 2 commits November 16, 2026 14:23
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The sweep and get_workflow_results_path honour ANALYZED_DATA_FOLDER, the
search stops at the stores, and the test no longer keeps its own copies of
the folder name, Zarr root files or marker name. Deduplicate the import
environment setup and the result-path table; require core 2.9.0b27.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@TorecLuik

Copy link
Copy Markdown
Collaborator Author

Independent review

Reviewed head a657704 (full re-review) against the #97 source dfc2243, with core from origin/analyzed-data-folder. Procedure and rules read from NL-BIOMERO master.

Verdict: approve after two should-fix items (one test each). No blocking findings.

Results

  • Full harness (this branch, BIOMERO_SCRIPTS_ROOT = Remove the input marker from results after import on every path #97 source): 516 passed, 20 skipped, matching the description. The one skip in this PR's file is the symlink test (Windows without link privilege); test_the_search_never_follows_links covers the same behaviour without links.
  • Temporary local merge of test-suite + this branch + origin/use-core-output-constants-tests (Tests: shared output names come from core constants #102 tests) against the Use the core script-output constants #101 source ec6b1a9: clean merges, 611 passed, 9 skipped (none skipped for a missing core).
  • tests/requirements.txt floor biomero>=2.9.0b27,<3 is the right harness floor for the reader and constants, but b27 is not on PyPI yet, so pip install -r (and CI) cannot resolve until core publishes it. The PR must stay behind that release. I installed core from the branch worktree instead.

Findings

Severity Where Finding
should fix tests/test_import_retry_storage.py (only the namespace line is added) Nothing pins that find_existing_result_storage uses the configured folder. Hardcoding '.analyzed' there survives every test. With a custom folder, a retry after a failed importer registration would not find the staged results and would retransfer. Add a test next to the existing ones that sets ANALYZED_DATA_FOLDER and expects results in <custom>/<wf> to be found (and a copy under .analyzed not).
should fix tests/test_input_marker_cleanup.py Same for the finalize lock folder in runScript (read_analyzed_data_folder() replaced by '.analyzed' survives). Cheapest fix: an AST-level assertion that neither script uses a string literal equal to the default folder name in code, like the existing output-name contract test; this also covers future hardcodes.
nit tests/test_input_marker_cleanup.py The autouse fixture that clears ANALYZED_DATA_FOLDER exists only in this file; test_import_retry_storage.py would behave differently on a host that sets it. Move it to a shared conftest.py.
nit tests/test_input_marker_cleanup.py, parametrize of test_refuses_the_storage_folder_and_a_workflow_folder The line is over 100 characters; wrap it.

Nothing found for: tests that restate the implementation or assert on their own mocks; the fake layout (make_store directly under <result folder>/<wf>/<ts>) matches the real archive layout; hang risk (the walk stubs are finite); skipping core-dependent tests (the imports fail loudly).

Mutation checks (scratch copy of #97 source, targeted files)

Killed: sweep compares to a literal .analyzed; depth limit removed; followlinks=True; scope depth off by one; get_workflow_results_path hardcoded; Results_Path output removed; depth-0 skip removed.
Survived: find_existing_result_storage hardcoded; finalize lock folder hardcoded (the two should-fix items).

🤖 Generated with Claude Code

…rals

Test that the retry lookup reads ANALYZED_DATA_FOLDER and that no script
hardcodes the default name (which also covers the finalize lock folder).
Share the env-clearing fixture in conftest.py; wrap a long line.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@TorecLuik

Copy link
Copy Markdown
Collaborator Author

Resolved in c7595d0: test_import_retry_looks_in_the_configured_result_folder pins the retry lookup, and test_no_script_hardcodes_the_default_result_folder_name (AST, both scripts) pins the finalize lock folder and any future literal. Both survived mutations now fail on a scratch copy of the #97 source (retry lookup hardcoded: 2 red; lock folder hardcoded: 1 red). The env-clearing fixture is in tests/conftest.py; the long parametrize line is wrapped. Harness against #97: 519 passed, 20 skipped.

🤖 Generated with Claude Code

TorecLuik and others added 5 commits November 16, 2026 16:07
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… sweep

The result folder is no longer configurable in 1.8 (NL-BIOMERO#111): the sweep
refuses a Results_Path of another workflow and accepts what the import makes,
and only SLURM_Import_Results.py names the folder, once. Core floor is
b26 again.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The retry-lookup tests exec find_existing_result_storage in a bare namespace.
Source that defines the RESULTS_FOLDER constant needs it there; source that
does not (master) is unaffected.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…-after-import

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rness branch

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@TorecLuik

Copy link
Copy Markdown
Collaborator Author

Independent review

Reviewer: independent agent. Head d24ff0a (contains #104) against #97 1133f5b.

Verdict: approve. No blocking. 1 should fix (small test gap), 3 nits.

Severity Where Finding
Should fix tests/test_input_marker_cleanup.py, test_only_the_import_script_names_the_results_folder_and_only_once The finalize lock folder in runScript (result_import_recovery(os.path.join(..., RESULTS_FOLDER, str(wf_id)), ...), SLURM_Import_Results.py:5433) is not covered: replacing RESULTS_FOLDER there by ".analyzed2" survives (M15), because the test only counts literals equal to .analyzed and runScript is not executed. Assert that the name RESULTS_FOLDER is loaded exactly three times in the import script (AST Name loads: result path, retry lookup, lock folder).
Nit same file, test_a_missing_result_folder_is_not_an_error Asserts only the return value, not the WARNING; dropping the isdir guard survives (M17) because the os.walk error callback also returns 0. Assert the "does not exist" warning.
Nit helper load_functions UUID is str and the id is "wf-1"; the real run passes a uuid.UUID. Add one case with a real UUID (the parent-name check uses str(workflow_id)).
Nit test_the_search_stops_at_the_stores_and_does_not_descend Pins that a store inside a subfolder keeps its marker; the importer accepts stores at any depth. Fine as a pin of the current design; see the depth finding on #97.

Mutation checks (scratch copy of #97 source; tests = this file plus test_import_retry_storage.py)

Mutation Result
RESULTS_FOLDER value changed killed (5 tests)
scope check always true killed (6)
followlinks=True killed (test_the_search_never_follows_links)
drop subdirectories.clear() killed (3)
sweep the result folder root too killed (11)
never call the cleanup after import killed (6)
cleanup also runs on a failed import killed (2)
cleanup exception not caught killed
no Results_Path output set killed (textual contract test)
ignore the shallow-storage flag killed (3)
never os.remove killed (10)
realpath replaced by abspath killed (symlinked result folder; junction on Windows)
retry lookup folder different killed (3)
result path folder different killed
empty-output warning downgraded to DEBUG killed
skip log at INFO instead of DEBUG killed (levels asserted exactly)
sweep depth 2 killed (2)
finalize lock folder different survived (should fix)
missing-folder guard dropped survived (nit)
name match loosened to startswith survived, equivalent mutation: the code still removes the exact name

Other checks

🤖 Generated with Claude Code

TorecLuik and others added 2 commits November 16, 2026 16:42
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ing and a real UUID

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TorecLuik and others added 4 commits November 16, 2026 16:49
…hallow-storage flag

Bare-namespace tests of the runner and image transfer now also provide
constants.results.CANONICAL_INPUTS_OUTPUT and SHALLOW_ZARR_STORAGE_ENABLED, and
the transfer output test accepts the core constant. Source that defines the
name or the expression locally (master) is unaffected.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…arker-after-import

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…_ENABLED

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…arker-after-import

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@TorecLuik

Copy link
Copy Markdown
Collaborator Author

Resolved in 41f6833: test_the_import_script_reads_the_results_folder_constant_three_times (AST: three loads of RESULTS_FOLDER; replacing the lock-folder use makes it red), the missing-folder test asserts the 'does not exist' warning (dropping the guard now turns it red), and a real-UUID case is added. Also merged #105 (harness namespaces).

🤖 Generated with Claude Code

@TorecLuik

Copy link
Copy Markdown
Collaborator Author

Independent review (follow-up)

Reviewed the delta d24ff0a..41f6833: the new commit b83c547 plus merges of test-suite and #105 (the merged content is the #105 diff reviewed there). Rules read from NL-BIOMERO master and biomero-scripts master. Core 2.9.0b26 from PyPI, one suite at a time.

Verdict: approve. No blocking or should-fix findings; 1 nit.

Checked

  • Harness against Remove the input marker from results after import on every path #97 head d0499b5: 520 passed, 35 skipped.
  • test_a_missing_result_folder_is_not_an_error now uses a valid <wf>/<timestamp> path, so the missing-folder branch (not the scope check) is reached, and asserts the WARNING.
  • test_the_workflow_id_is_compared_as_text_for_a_real_uuid passes a real uuid4, as runScript does.
  • test_the_import_script_reads_the_results_folder_constant_three_times counts AST loads of RESULTS_FOLDER for the three uses (result path, retry lookup, finalize lock folder in runScript, which no test executes).
  • The setOutput contract assertion normalises whitespace so the wrapped call matches; it still checks the exact call text.

Mutation checks (scratch copy of the #97 source)

Mutation Killed by
missing-folder warning lowered to DEBUG test_a_missing_result_folder_is_not_an_error
workflow id compared without str() test_the_workflow_id_is_compared_as_text_for_a_real_uuid only (the other tests pass strings, so this test is needed)
finalize lock folder hardcoded ".analyzed" ..._only_once and ..._three_times
retry lookup hardcoded ".analyzed" ..._only_once and ..._three_times
setOutput publishes WORKFLOW_UUID_OUTPUT instead of RESULTS_PATH_OUTPUT test_both_scripts_use_the_core_constant_for_the_result_folder_output

Findings

Severity Where Finding
nit tests/test_input_marker_cleanup.py:619-627 The count of exactly three loads is an AST proxy for "the finalize lock uses the constant": a legitimate fourth use fails it, and swapping one use for another constant would keep three. It is the best option while no test runs runScript; the comment says so.

None found for the other rule categories (test-only PR).

Needs a real stack

Unchanged from the previous review.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant