Repository navigation
Conversation
- Import the Zarr contracts from biomero_schema.zarr; biomero.zarr_contracts is removed from core. - Import the metadata view functions from biomero.metadata_views (renamed from biomero.provenance) and format reduced values with core's format_large_field_reference, so core recognises them on refresh. - Move the Blueprint workflow color to SLURM_Run_Workflow.py, which forwards the resolved color to the result scripts. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…lation in the scripts - Registration builds the launcher params from the script client and hands them to WorkflowTracker.add_launcher_task (core only sets the marker). - Resume reads the task aggregates (completed/failed) instead of a second source, raises when the history cannot be read, never re-attaches to a failed/cancelled/timed-out Slurm job, and skips a workflow whose results were already imported instead of submitting it again. - The pipeline stops between steps and between polls when a cancel was requested, cancels its Slurm jobs, closes its tasks and fails the workflow. - One reader of BIOMERO_DETACHED_WORKFLOWS: biomero.detached. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Fail with a clear message when the installed biomero predates biomero.metadata_views; README states the minimum core release. - Move the colour palette to the module constants area of the runner. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…stages split - WorkflowCancelled raised inside export, conversion and submission is no longer turned into a generic error by those stages' handlers. - cancel_workflow_run fails every open task of the workflow (not only the Slurm-job tasks) and cancels the jobs found in the task history. - admin/SLURM_Cancel_Workflow.py requests a cancel for a workflow id (owner or administrator), so a detached run can be cancelled end to end. - run_pipeline_stages split into named stages that return their state. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…neRun, monitor split - Stage handlers re-raise any WorkflowInterrupted without failing tasks; only WorkflowCancelled makes the run cancel jobs and fail tasks and workflow. - execute_workflow_pipeline(stop_check=...) is an explicit hook for the caller (a detached worker that lost its claim); stage arguments grouped in PipelineRun. - monitor_slurm_jobs split into per-state handlers; one failed-job path. - cancel_workflow_run fails tasks and the workflow even when scancel raises. - Cancel script: finished workflows record nothing, reason bounded, denial logged once, plain copyright header. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…, nits - PipelineRun.raise_if_claim_lost: checked after the poll's progress read and before the outcome write (a late cancel still completes); a stop check also between conversion and submission. - cancel_workflow_run: reading the tasks is inside the scancel handling; a failure puts 'Slurm jobs [...] could not be cancelled' into the reason. - Dead handler in run_workflow removed; run parameter renamed pipeline; types and docstring for tracker. - Cancel script message mentions inline batched runs do not honour cancel. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The workflow runner now owns the cleanup of .biomero-input.json: after the result import succeeded it removes the marker from the imported result folder. SLURM_Import_Results.py reports that folder as the Results_Path output. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
biomero.zarr_contracts is being removed from core; the marker constant is defined in biomero_schema.zarr, which the scripts already import. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ogether Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The harness still expects the CanonicalInput import from biomero.zarr_contracts until #88 and its tests merge; only the new TRANSFER_INPUT_MARKER import comes from biomero_schema.zarr. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- The admin migration passes the importer's PROCESSED_DATA_FOLDER explicitly, moves sidecars with move_canonical_marker and cleans up with remove_external_canonical_markers (no empty .biomero folders left). - upgrade_reused_canonical no longer fails the workflow when the BIOMERO folder is not writable: it logs a warning and keeps the registration unverified. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Accept only .analyzed/<workflow>/<timestamp>, never fail the run on a cleanup error, warn on unreadable folders, drop unreachable parameters, name the shallow storage condition and document the Results_Path output. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Extracted from runScript so the persisted launcher params, the contract with the supervisor's batched parent, can be tested. No behaviour change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Both scripts take RESULTS_PATH_OUTPUT from core (biomero#110), like WORKFLOW_UUID_OUTPUT, instead of defining it locally. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Producers and readers of the Message, URL, File_Annotation and Canonical_Inputs outputs use biomero.constants.results instead of their own string literals. Results_Path is handled by the result-path PR. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- upgrade_reused_canonical treats a corrupt, uncommitted or unreachable sidecar (OSError, InvalidCanonicalStore, ValueError) as absent, with a WARNING naming the path and reason. - The transfer script passes the importer's PROCESSED_DATA_FOLDER to every sidecar call, as the admin migration already does. - _move_canonical_store renames the store back, or raises a clear error, when moving its sidecar fails after the rename. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Resolve the gated import block with #97 (the input marker cleanup, which merges first) to `from biomero_schema.zarr import CanonicalInput, TRANSFER_INPUT_MARKER`. Name the released minimum core version (2.9.0b26) in the README and in the import error messages. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Conflicts with the PRs that merge before this one, resolved by taking their code and applying the same mechanical replacement of output-name literals with biomero.constants.results.*_OUTPUT: - SLURM_Run_Workflow.py: keep RESULTS_FOLDER_NAME and ZARR_ROOT_FILES from #97, drop the local CANONICAL_INPUTS_OUTPUT; the export, conversion and import stages of #90 use MESSAGE_OUTPUT, URL_OUTPUT and FILE_ANNOTATION_OUTPUT. - SLURM_Run_Workflow_Batched.py: use #90's register_detached_batches and MESSAGE_OUTPUT for the launcher message. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…er>=1.5.0b14 The merge is clean. The scripts declared no importer minimum anywhere, so the README now names biomero-importer>=1.5.0b14 (it provides the PROCESSED_DATA_FOLDER and sidecar helpers this PR imports). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e stores The result folder is the ANALYZED_DATA_FOLDER setting (read_analyzed_data_folder), not a local constant. The marker sits at the root of each exported store, one level below the result folder, so the sweep no longer detects Zarr stores. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Scripts and core are released together, so the README and the import error name the renamed module, not a version floor. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Independent reviewScope: only the forward-merge commit ec6b1a9 ("Merge #95 into use-core-output-constants"), compared with both parents (9def6e0 and a19f4d7) via Verdict: the merge is correct. No blocking findings.
Nothing found in any category for the merge. 🤖 Generated with Claude Code |
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…constants Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Resolved in d34083c: #95 (160938c) merged in, clean. 🤖 Generated with Claude Code |
…the folder name Core will not provide an ANALYZED_DATA_FOLDER setting for 1.8 (tracked in NL-BIOMERO#111). SLURM_Import_Results.py keeps one RESULTS_FOLDER constant, and the marker sweep accepts only a result folder of this run's workflow: <...>/<workflow id>/<timestamp>. No general constants remain in the run script. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…constants Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Independent reviewReviewer: independent agent. Scope: the latest forward-merge commit of this branch (708e3da) only (conflict resolution), plus a repository-wide search; the PR's own change was reviewed in its original review. Verdict: approve the forward merge. No blocking, no should fix.
🤖 Generated with Claude Code |
…d exact scope-check wording Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…constants Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Resolved in 963101f: #95 (9573c42) merged in; the only conflict was line wrapping of the same core-constant lines, resolved to the wrapped form. 🤖 Generated with Claude Code |
Closes #100
Refs NL-BioImaging/biomero#110
Core: the scripts need the matching core release (it contains biomero#111); scripts and core are released together. No runtime check is added to each script. The
VERSIONof every script is locked to the core release and is bumped in the release preparation, not in this PR.Conflicts resolved in this branch (it contains #97, #88, #90 and #95, which merge before it):
SLURM_Run_Workflow.py(kept #97'sRESULTS_FOLDER_NAME/ZARR_ROOT_FILES, dropped the localCANONICAL_INPUTS_OUTPUT, took #90's stage functions and applied the constants to their new literals) andSLURM_Run_Workflow_Batched.py(took #90'sregister_detached_batches,MESSAGE_OUTPUTfor the launcher message). Dead commented-outsetOutput("File_Annotation", ...)lines removed; touched long lines reflowed.Producers and readers of the outputs
Message,URL,File_AnnotationandCanonical_Inputsusebiomero.constants.results.MESSAGE_OUTPUT,URL_OUTPUT,FILE_ANNOTATION_OUTPUTandCANONICAL_INPUTS_OUTPUTinstead of string literals, inSLURM_Run_Workflow.py,SLURM_Run_Workflow_Batched.py,SLURM_CellPose_Segmentation.py,_SLURM_Image_Transfer.py,SLURM_Get_Results.py,SLURM_Import_Results.py,SLURM_Get_Update.py,SLURM_Remote_Conversion.pyand_SLURM_File_Transfer.py. The two localCANONICAL_INPUTS_OUTPUTdefinitions are removed.Results_Pathis left to scripts#97. The"type": "URL"of the OMERO URL value and the internalUI_messageskeys of the batched script are not output names and stay. The admin scripts only show a message to the user (no reader) and are unchanged.No behaviour change: the strings are identical.
Tests
Scripts tests live on
test-suite: companion PR #102 adds a contract test that fails if a script uses the literal, and gives the existing tests that run script functions in a bare namespace the constants. Harness (test-suite with #102) against this branch: 606 passed, 9 skipped, core 2.9.0b26 from PyPI. The contract test flags every string constant equal to one of the four names (except docstrings,"type": "URL"and the batched runner'sUI_messageskeys).Merge order (production and test PRs, interleaved)
Each step keeps
masterCI green: every (production state, test-suite state) pair on this path was run locally with core 2.9.0b26 from PyPI (482, 484, 484, 513, 524, 588, 594, 597, 606 passed; no failures). The production PRs merge cleanly in this order: each branch already contains the conflict resolutions with the PRs before it.test-suite:biomero>=2.9.0b26,<3intests/requirements.txt, so core-dependent tests run in CI)masterand for the new source)masterbefore Remove the input marker from results after import on every path #97, by design)A production PR's own check stays red until the test PRs before it in this list are merged (CI runs the shared harness from
test-suite, which is not pinned); merge the test PR, then rerun the check.🤖 Generated with Claude Code