Skip to content

Fix GCP Batch co-execution deadlocks#466

Merged
mvdbeek merged 4 commits into
galaxyproject:masterfrom
ksuderman:gcp-batch-coexecution-fixes
Jul 3, 2026
Merged

Fix GCP Batch co-execution deadlocks#466
mvdbeek merged 4 commits into
galaxyproject:masterfrom
ksuderman:gcp-batch-coexecution-fixes

Conversation

@ksuderman

Copy link
Copy Markdown
Contributor

Part of #465.

This PR ensures the tool and sidecar runnables are executed in the correct order and don't deadlock.

This PR also:

  1. Ensures outputs from intermediate jobs are collected.
    Galaxy's path rewriting causes tools to write outputs into working/ rather than outputs/, but __collect_outputs() only checked the outputs directory, so those files were skipped. Also relaxes the dynamic collection pattern from dataset_\d+\.dat to dataset_[\w-]+\.dat so UUID-named datasets (e.g. dataset_b0e57817-…​.dat) match.
  2. Re-raises infrastructure errors so they fail the job rather than producing a "successful" job with empty outputs. FileNotFoundError is deliberately excluded.
  3. Pass docker_extra_volumes to the Batch containers
    Comma-separated Docker -v-style volumes (used to bind-mount CVMFS paths such as
    /cvmfs/data.galaxyproject.org) are now applied to both the sidecar and tool runnable
    containers, so tools that depend on CVMFS reference data work.

@mvdbeek

mvdbeek commented Jun 19, 2026

Copy link
Copy Markdown
Member

Could you pull out d2487a8 and 7244d2c into separate PRs ? I'm happy to merge the rest

@ksuderman ksuderman force-pushed the gcp-batch-coexecution-fixes branch from d2487a8 to bcd073e Compare June 19, 2026 16:26
@ksuderman

Copy link
Copy Markdown
Contributor Author

Could you pull out d2487a8 and 7244d2c into separate PRs ? I'm happy to merge the rest

I will open another PR with these two commits in a moment.

@mvdbeek mvdbeek merged commit 940cf3e into galaxyproject:master Jul 3, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants