qxub

qxub is a sophisticated PBS job submission wrapper designed for HPC environments. It eliminates the boilerplate and complexity of writing PBS job scripts by providing a unified command-line interface for submitting jobs across different execution contexts. Whether you need to run code in conda environments, with environment modules, inside containers, or as direct submissions, qxub handles the PBS integration seamlessly while offering intelligent queue selection, resource management, and comprehensive configuration options.

Submit PBS jobs with conda environments, modules, or containers.

Install

git clone https://github.com/swarbricklab/qsub_tools.git
cd qsub_tools
pip install -e .

Usage

# Basic syntax: qxub exec [options] -- command
qxub exec --env myenv -- python script.py        # Conda environment
qxub exec --mod python3 -- python script.py     # Environment module
qxub exec --sif container.sif -- python script.py  # Singularity container
qxub exec -- python script.py                    # Direct submission

# Built-in command aliases for faster typing
qx dvc push                                      # Short for 'qxub exec -- dvc push'
qxi --env pytorch                                # Short for 'qxub interactive --env pytorch'
qxtat 12345.gadi-pbs                             # Short for 'qxub status'
qxet "command" --env base                        # Short for 'qxub config shortcut set "command" --env base'

# Interactive PBS sessions with environment activation
qxub interactive --env pytorch                   # Start interactive session
qxi --env pytorch --tmux ml-session              # With tmux persistence
qxi --record --env pytorch                       # Record session for replay

# Replay recorded sessions
qxub replay --list                               # List recordings
qxub replay 1                                    # Replay most recent
qxub replay --view 1                             # View transcript

# Resource specification - choose your style!
qxub exec --mem 16GB --cpus 8 --runtime 2h --env myenv -- python script.py        # Workflow-friendly
qxub exec --resources mem=16GB,ncpus=8,walltime=2:00:00 --env myenv -- python script.py  # Traditional PBS

# Cost-optimized auto queue selection (recommended!)
qxub exec --queue auto --mem 1200GB --env myenv -- python big_job.py    # → megamem (58% cheaper!)
qxub exec --queue auto --cpus 5000 --env myenv -- python parallel.py    # → normalsr

# GPU jobs - queue and CPUs are set automatically
qxub exec --gpus 1 --env pytorch -- python train.py                     # → gpuvolta, 12 CPUs
qxub exec --gpus 4 --gpu-type a100 --env pytorch -- python train.py     # → dgxa100, 64 CPUs

# Preview without running
qxub exec --dry --env myenv -- python script.py

# Shortcuts - automatic command detection
qxub exec -- python script.py     # Auto-detects 'python' shortcut (conda: base)
qxub exec -- gcc --version        # Auto-detects 'gcc' shortcut (modules: gcc)

# Explicit shortcut usage
qxub exec --shortcut python -- script.py

Key Rules

• All qxub exec options go before --
• Your command goes after --
• Use --dry to preview
• Commands starting with known prefixes trigger shortcuts automatically

Options

Option	Description	Example
--env	Conda environment	--env pytorch
--mod	Environment module	--mod python3
--sif	Singularity container	--sif container.sif
--shortcut	Use predefined shortcut	--shortcut python
--cmd	Complex command (alternative to --)	--cmd "echo \"Hello ${USER}\""
--resources	PBS resources (traditional format)	--resources mem=16GB --resources ncpus=8
--mem / --memory	Memory requirement (workflow-friendly, default: configured)	--mem 16GB or --memory 4GB
--cpus / --threads	CPU cores/threads (workflow-friendly, default: configured)	--cpus 8 or --threads 4
--runtime / --time	Walltime limit (workflow-friendly, default: configured)	--runtime 2h30m or --time 1h
--disk / --jobfs	Local disk requirement (workflow-friendly, default: configured)	--disk 100GB or --jobfs 50GB
--gpus	Number of GPUs (auto-selects queue and CPUs)	--gpus 1 or --gpus 4
--gpu-type	GPU type: v100 (default) or a100	--gpu-type a100
--volumes / --storage	NCI storage volumes to mount (default: configured)	--volumes gdata/a56 or --storage gdata/a56+scratch/a56
--var	Set environment variable in job (repeatable)	--var FOO=bar --var BAZ=qux
--vars	Set multiple env vars (comma-separated)	--vars "FOO=bar,BAZ=qux"
--queue	PBS queue (use auto for cost optimization!)	--queue auto
--name	Job name	--name myjob
--project	PBS project	--project a56
--out	Output file path	--out /scratch/job.out
--err	Error file path	--err /scratch/job.err
--quiet	Silent monitoring (no progress messages or job ID)	--quiet
--terse	Emit job ID then silent monitoring	--terse
--remote	Execute on remote HPC system via SSH	--remote gadi
--dry	Preview without submission	--dry
-v/-vv/-vvv	Verbosity levels	-vv

Shortcuts System

# List available shortcuts
qxub config shortcut list

# Show shortcut source information (system vs user)
qxub config shortcut list --show-origin

# Create user shortcuts (default)
qxub config shortcut set "python" --env base --description "Python with base conda environment"
qxub config shortcut set "gcc" --mod gcc --description "GCC compiler with modules"

# Create system-wide shortcuts (admin/team use)
qxub config shortcut set "dvc data status" --system --env dvc3 --resources mem=64GB,ncpus=16

# Use shortcuts (automatic detection)
qxub exec -- python script.py      # Detects 'python' shortcut
qxub exec -- gcc --version         # Detects 'gcc' shortcut

# Explicit shortcut usage
qxub exec --shortcut python -- script.py

# Show shortcut details with source info
qxub config shortcut show python

# Delete shortcuts
qxub config shortcut delete python              # Delete user shortcut
qxub config shortcut delete "system-wide" --system --yes  # Delete system shortcut

# Rename shortcuts
qxub config shortcut rename "old-name" "new-name"
qxub config shortcut rename "old-name" "new-name" --system  # Rename system shortcut

# Built-in command aliases for faster access
qx --env myenv -- python script.py    # Equivalent to 'qxub exec'
qxtat 12345.gadi-pbs                   # Equivalent to 'qxub status'
qxet "ml-pipeline" --env pytorch       # Equivalent to 'qxub config shortcut set'

## Parallel Job Execution & Monitoring
```bash
# Submit multiple jobs and monitor them
find -name "*.csv" -exec qxub exec --terse --env myenv -- process.py {} \; | qxub monitor

# Monitor with live status display (emoji indicators)
qxub monitor 12345.gadi-pbs 12346.gadi-pbs 12347.gadi-pbs

# Summary mode for many jobs
echo -e "job1.gadi-pbs\njob2.gadi-pbs\njob3.gadi-pbs" | qxub monitor --summary

# Clean display without PBS suffix
qxub monitor --suffix .gadi-pbs 12345.gadi-pbs 12346.gadi-pbs

# Quiet monitoring for scripting
echo "12345.gadi-pbs" | qxub monitor --quiet

Job Management

# View execution history
qxub history executions --limit 10
qxub history latest

# View job output files (v3.2.0)
qxub history out                 # Most recent job stdout
qxub history err                 # Most recent job stderr
qxub history log                 # Most recent job PBS log
qxub history out 12345.gadi-pbs  # Specific job by ID

# View resource efficiency
qxub resources list --limit 5
qxub resources stats

# Configuration management
qxub config get defaults.project
qxub config set defaults.project "a56"
qxub config files

Configuration (Optional)

# Set personal defaults to avoid repetition
qxub config set defaults.project "a56"
qxub config set defaults.queue "normal"

# Create user shortcuts for personal workflows
qxub config shortcut set gpu --env pytorch --description "GPU training with PyTorch"
qxub exec -- gpu train.py    # Use the shortcut

# Create system-wide shortcuts for team use (requires admin permissions)
qxub config shortcut set "dvc repro" --system --env dvc3 --resources mem=32GB --description "DVC data pipeline"

# View configuration and shortcut files
qxub config files
qxub config shortcut files    # Shows both system and user shortcut file locations
qxub config get defaults

Help

qxub --help                       # General help
qxub exec --help                 # Execution command help
qxub interactive --help          # Interactive session help
qxub replay --help               # Session replay help
qxub config --help               # Configuration help
qxub config shortcut --help      # Shortcut management help
qxub history --help              # History management help
qxub resources --help            # Resource tracking help
qxub monitor --help              # Monitor multiple jobs

# Built-in alias help
qx --help                        # qxub exec help
qxi --help                       # qxub interactive help
qxtat --help                     # qxub status help
qxet --help                      # qxub config shortcut set help