Skip to content

guardrails runner: report deeply nested configs as exit 2 instead of crashing (fixes #400) - #410

Open
vanshwadhwa24 wants to merge 2 commits into
FinanceFlash:mainfrom
vanshwadhwa24:fix/guardrails-runner-recursion-error
Open

vanshwadhwa24 wants to merge 2 commits into
FinanceFlash:mainfrom
vanshwadhwa24:fix/guardrails-runner-recursion-error

Conversation

@vanshwadhwa24

Copy link
Copy Markdown

Problem and scope

guardrails_runner.py check crashes with an uncaught RecursionError (traceback, exit code 1) on deeply nested configuration JSON, breaking the documented contract: exit 2 for file/schema parsing failures, always a JSON report on stdout (reported in #400). A CI gate cannot distinguish a crashed parser from a failed validation. Scope is the runner's error handling plus regression tests.

What changed

  • guardrails_runner.py: main() now catches RecursionError alongside the existing OSError, ValueError, TypeError, so a deeply nested config produces the standard JSON error report on stdout and exit code 2. In my reproduction the RecursionError is raised by json.loads during parsing; the same handler would also catch one raised later in the command path, though I have only reproduced the parsing case.
  • Added test_guardrails_runner.py next to the runner with two regression tests: a 10,000-level config (60,001 bytes, under the 1 MB limit) must exit 2 with a parseable JSON report and no traceback on stderr, and the shipped guardrails_chatbot.json profile must still pass with exit 0.

Verification

  • python -m black --config .github/quality/pyproject.toml --check .github/quality/tests
  • python -m flake8 --config .github/quality/.flake8 .github/quality/tests
  • python -m pytest -c .github/quality/pyproject.toml .github/quality/tests (924 passed; this suite does not discover the new test file)
  • I used UnvibeCode on a representative repository when the change affects code analysis, CLI behaviour, engineering guidance, or code-specific claims. (Left unchecked deliberately: the change is a one-line error-handling fix in a shipped example script, and the code-specific claims in this PR are verified by the direct test runs below rather than by an UnvibeCode review, which would not exercise this behavior.)
  • I updated documentation for changed user-facing behaviour. (Not applicable: this restores the already-documented exit-2 contract.)
  • I included no credentials, proprietary source, customer data, generated customer reports, or unsanitized logs.

Changed-file checks: the new test_guardrails_runner.py passes black --check and flake8 with the repo configs. The modified guardrails_runner.py is a one-line change to an existing statement, left in the file's current style; note CI currently lints only .github/quality/tests, so neither skills/ file is covered by CI lint (#402 tracks that gap).

Direct test runs (from skills/unvibecode_gaurdrails_implemenataion_pack/Pre-configured_gaurdrails_programs/, python -m pytest test_guardrails_runner.py -q):

  • On unpatched main: 1 failed, 1 passed - the deeply nested config (60,001 bytes, 10,000 levels) crashes with RecursionError: maximum recursion depth exceeded while decoding a JSON object, traceback on stderr, exit code 1, no stdout report.
  • On this branch: 2 passed - the same config prints {"configuration_valid": false, "runtime_verified": false, "error": "Invalid configuration or file operation; check the schema and path."} on stdout and exits 2.

Disclosure: this change was prepared with AI assistance (code edit and test runs); the before/after results above are the actual outputs from the local runs.

Fixes #400

…crashing

json.loads raises RecursionError on deeply nested configuration, which escaped the error handler in main() and produced an uncaught traceback with exit code 1, breaking the documented contract (exit 2 and a JSON report on stdout for parsing failures). Catch RecursionError alongside the existing exceptions.
Cover the deeply nested config contract (exit 2 with a JSON report, no traceback) and the shipped chatbot profile passing with exit 0. The deeply nested case fails on the unpatched runner.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Guardrails runner — deeply nested config JSON crashes check with an uncaught RecursionError and exit code 1 instead of the documented exit 2

1 participant