AGENTS.md

Python tests

• Read guidance for the production code. Use existing suites (tests/app/, tests/backend/, or the owning root suite) and matching conftest.py/fixtures/ fixtures. UI copy and docs checks belong to their packages.
• Tests that read files outside tests/, scripts/, or the Python package tree: add those paths to .github/workflows/python-tests.yml change filters, so fixture-only changes trigger pytest.
• Name tests by outcome. Cover meaningful edge and failure cases. Survive behavior-preserving refactors. Prefer real temporary SQLite/filesystem/tensor operations. Mock external services, expensive models, unavailable hardware; never mock the transaction, authorization, concurrency, or numerical behavior under test.
• Isolate paths, databases, accounts, clocks, seeds, environment, network behavior. Clean up subprocesses, listeners, patches, resources. Never use production model/database/output directories.
• Cover affected rollback, revisions, cross-user isolation, retries, cancellation, bounds, cleanup. IndexedDB, Web Locks, focus, rendering belong in webv2 browser tests.
• Use root pytest commands. CI runs one xdist worker per vCPU; windows-ffmpeg runs two. --dist loadfile in addopts keeps each file on one worker; file assignment and order are arbitrary. Each file sets up its own process-global state (e.g. JWT secret) and passes alone. Verify new files on their own.
• Default tests assert behavior, not throughput or latency under runner contention. Generous hang ceilings fine. CPU time tolerates contention better than wall time. Hardware/quiet-machine measurements need slow, excluded by default and in CI. Dev machines: -m slow for that lane, -m "" for everything.
• Keep a test's run time to the work it checks; idle seconds add up across the suite, and the ffmpeg tests run on two workers in windows-ffmpeg. Check new files with --durations=10, which splits setup, call and teardown.
  • No real waits. Poll intervals, retry backoffs and grace periods belong in module-level constants that a fixture can shorten. The root conftest.py already shortens the download, installer and image-index queue polls, so a service's stop() in teardown returns at once. Shorten a retry backoff only in the tests that script its failures, as _fake_move in test_model_install.py does: the backoff is what lets a real Windows file handle close.
  • Passwords hash at bcrypt cost 4 in tests. Request production_password_hashing to assert or measure the production cost.
  • A fresh interpreter costs about ten seconds of imports. Spawn one only when isolation is the point (import order, sys.modules, process-global state), and check many cases inside one subprocess unless each case needs its own cold start.
  • A full gc.collect() walks the worker's whole heap, a third of a second late in a session. Tests that collect repeatedly freeze the pre-existing heap first, as test_model_cache_ram_budget.py does.
• Mark a test platform_independent only when no operating system can change its outcome for the same dependency set: import-graph boundaries in a fresh interpreter, static source scans, deterministic counts or shapes. Windows CI, which installs the linux dependency set, deselects them; macOS runs a different torch and keeps them. Numerics (BLAS kernels differ per platform), process-spawning behavior (start methods, helper executables), filesystem paths and locking, and dependency smoke tests stay unmarked.
• Mark a test ffmpeg when it runs an ffmpeg binary or loads the libav libraries, directly or through fixtures and helpers, including tests that feed ffmpeg bad input to check its rejection. Not when ffmpeg is mocked, or only spawned incidentally on input every decoder rejects. The windows-cpu runner is Windows Server Core, whose missing video DLLs stop every standard ffmpeg build from loading, so it deselects them; windows-ffmpeg runs only them on a hosted runner. Module-level pytestmark when the whole file needs it. To find unmarked ones, reproduce that runner: unset IMAGEIO_FFMPEG_EXE, rename the binary under imageio_ffmpeg/binaries/ in the venv so get_ffmpeg_exe() raises (with no ffmpeg on PATH), set OPENCV_VIDEOIO_PRIORITY_FFMPEG=0, and run -m "not slow and not ffmpeg". Pointing IMAGEIO_FFMPEG_EXE at a failing program is not enough: imageio-ffmpeg returns that path untested.
• State hardware prerequisites and skipped/unavailable devices. CPU or mocked passes do not prove CUDA/ROCm/MPS/XPU behavior.
• No CPU bf16 matmul unless low precision is the behavior under test. Why: torch's CPU bf16 GEMM faults with an illegal instruction (0xc000001d) on part of the hosted windows fleet; that aborts the worker and fails the job. Run the forward at float32 and assert the width separately, as the nvfp4 loader tests do. Skip on win32 only when bf16 itself is the subject.