feat(#2095): add verified/unchecked variable fields to review findings schema by fullsend-ai-coder[bot] · Pull Request #2363 · fullsend-ai/fullsend

fullsend-ai-coder · 2026-06-16T20:04:11Z

Add optional verified_variables and unchecked_variables array fields to the finding object in review-result.schema.json. These fields force the review agent's security sub-agent to structurally enumerate which variables it verified as sanitized and which it did not, preventing blanket safety claims from partial verification.

Changes:

review-result.schema.json: add both arrays (string items, minLength 1)
to the finding definition under additionalProperties: false
security.md (sub-agent): update verification methodology step 3 to
require populating the new structured fields instead of prose
code-review/SKILL.md: update exhaustive security-control verification
guidance to reference the new fields
agents/review.md: add new fields to the finding object documentation
table
validate-output-schema-test.sh: add three test cases (partial
coverage valid, full coverage valid, empty-string variable rejected)
and update the allowed-properties assertion

Pre-commit could not run (shellcheck install failed due to network restrictions in the sandbox). The post-script runs pre-commit authoritatively.

Closes #2095

Post-script verification

Branch is not main/master (agent/2095-structured-variable-enumeration)
Secret scan passed (gitleaks — 32f73a4f93301493d2c31be3970aa4c51a26acc7..HEAD)
Pre-commit hooks passed (authoritative run on runner)
Tests ran inside sandbox

github-actions · 2026-06-16T20:04:24Z

E2E tests did not run

E2E tests run automatically for org/repo members and collaborators on pull requests.

For other contributors, a maintainer must add the ok-to-test label after the latest push.

See E2E testing guide for details.

github-actions · 2026-06-16T20:05:48Z

Site preview

Preview: https://43bfc712-site.fullsend-ai.workers.dev

Commit: 837e95fe362cf46411321af17de167e8137bba19

fullsend-ai-review · 2026-06-16T20:07:52Z

🤖 Finished Review · ✅ Success · Started 8:07 PM UTC · Completed 8:20 PM UTC
Commit: a34d13c · View workflow run →

codecov · 2026-06-16T20:12:16Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

fullsend-ai-review · 2026-06-16T20:19:58Z

Review

Findings

Critical

[breaking-schema-change] internal/scaffold/fullsend-repo/schemas/review-result.schema.json:56 — The schema makes verified_variables and unchecked_variables required on every finding (additionalProperties: false), but multiple consumer templates and the Go struct have not been updated. Specifically: (1) Three finding templates in SKILL.md (lines 380–389, 405–412, 537–544) omit the fields. (2) The meta-prompt.md template (lines 14–24) injected into every sub-agent's system prompt omits them. (3) The challenger.md template (lines 48–59) omits them. (4) The review-result-label-actions-test.sh test at line 72 constructs a finding without the fields and expects validation to pass. (5) The Go ReviewFinding struct in internal/cli/postreview.go:166-174 lacks both fields. After this PR merges, all agents following any of these templates will produce findings that fail schema validation, triggering harness retry loops per ADR 0022.
Remediation: Either (a) make the two new fields optional (remove from required array) and rely on documentation/instructions to ensure security sub-agents populate them, or (b) update ALL consumers atomically in this PR: add the fields to all three SKILL.md templates, meta-prompt.md, challenger.md, the label-actions test, and the Go struct.

Previous run

Review

Findings

Critical

[breaking-schema-change] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — Adding verified_variables and unchecked_variables as required fields (combined with additionalProperties: false) breaks all consumers that produce findings without these fields. Multiple template files within the same scaffold (SKILL.md, meta-prompt.md, challenger.md) and the label_actions test file construct findings without the new required fields. The Go ReviewFinding struct in internal/cli/postreview.go also lacks these fields. After this PR merges, deployed review agents will produce findings that fail schema validation, triggering harness retry failures.
Remediation: Either make the new fields optional rather than required, or update ALL consumers atomically in this PR (SKILL.md templates, meta-prompt.md, challenger.md, label-actions-test.sh, and the Go struct).

High

[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:380 — The standard finding format template (lines 380–389) and the sub-agent failure finding templates (lines 405–412 and 537–544) do not include verified_variables or unchecked_variables. Agents following these templates will produce findings that fail schema validation.
Remediation: Add "verified_variables": [] and "unchecked_variables": [] to all three finding templates in SKILL.md.
[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/meta-prompt.md:15 — The meta-prompt finding format template (lines 14–24) is injected into every sub-agent's system prompt and does not include the new required fields. Sub-agents following this template will produce findings missing the required fields.
Remediation: Add "verified_variables": [] and "unchecked_variables": [] to the finding template in meta-prompt.md.

Medium

[consumer-completeness] internal/scaffold/fullsend-repo/schemas/review-result-label-actions-test.sh:72 — The label_actions test file constructs a finding (lines 72–77) without the new required fields. This test expects validation to pass but will fail after this PR merges.
Remediation: Add "verified_variables": [] and "unchecked_variables": [] to the finding object.
[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/challenger.md:48 — The challenger sub-agent's adjudicated_findings output format template does not include the new required fields. The challenger reshapes findings and may drop these fields if the template doesn't show them.
Remediation: Add the fields to the adjudicated_findings template.

Previous run

Review

Findings

Medium

[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:380 — The standard finding format template (lines 380–389) and the sub-agent failure finding template (lines 405–412) do not include verified_variables or unchecked_variables. After this PR merges, the schema requires both fields on every finding (additionalProperties: false). Sub-agents and the orchestrator following these templates will produce findings that fail schema validation, triggering harness retry iterations.
Remediation: Add "verified_variables": [] and "unchecked_variables": [] to both finding templates.

Previous run (2)

Review

Findings

Low

[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/challenger.md:48 — The challenger sub-agent's output format template does not include verified_variables or unchecked_variables. The challenger passes through input findings and the orchestrator strips only challenger-specific fields, so data is not lost in practice. Updating the example would improve documentation completeness.
[incomplete-doc] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:378 — The finding object format example in SKILL.md step 5 omits verified_variables and unchecked_variables. Sub-agents that need these fields already have explicit instructions in their own skill files.
[incomplete-doc] internal/scaffold/fullsend-repo/skills/pr-review/meta-prompt.md:12 — The output format section in meta-prompt.md does not include verified_variables and unchecked_variables. Sub-agents that need these fields receive explicit instructions in security.md and code-review/SKILL.md.
[requirement-escalation] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/security.md — Documentation states both arrays "must be present" in certain contexts while the schema marks them as optional. This is a deliberate pattern: schema-level optionality with documentation-level requirements for specific contexts.
[consumer-completeness] internal/cli/postreview.go:159 — The Go ReviewFinding struct does not include verified_variables or unchecked_variables fields. json.Unmarshal will silently discard them. No current logic uses these fields.

Labels: PR modifies review agent schema and security sub-agent verification methodology

Previous run

Review

Findings

Critical

[breaking-schema-change] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — Adding verified_variables and unchecked_variables as required fields (combined with additionalProperties: false) breaks all consumers that produce findings without these fields. Multiple template files within the same scaffold (SKILL.md, meta-prompt.md, challenger.md) and the label_actions test file construct findings without the new required fields. The Go ReviewFinding struct in internal/cli/postreview.go also lacks these fields. After this PR merges, deployed review agents will produce findings that fail schema validation, triggering harness retry failures.
Remediation: Either make the new fields optional rather than required, or update ALL consumers atomically in this PR (SKILL.md templates, meta-prompt.md, challenger.md, label-actions-test.sh, and the Go struct).

High

[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:380 — The standard finding format template (lines 380–389) and the sub-agent failure finding templates (lines 405–412 and 537–544) do not include verified_variables or unchecked_variables. Agents following these templates will produce findings that fail schema validation.
Remediation: Add "verified_variables": [] and "unchecked_variables": [] to all three finding templates in SKILL.md.
[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/meta-prompt.md:15 — The meta-prompt finding format template (lines 14–24) is injected into every sub-agent's system prompt and does not include the new required fields. Sub-agents following this template will produce findings missing the required fields.
Remediation: Add "verified_variables": [] and "unchecked_variables": [] to the finding template in meta-prompt.md.

Medium

[consumer-completeness] internal/scaffold/fullsend-repo/schemas/review-result-label-actions-test.sh:72 — The label_actions test file constructs a finding (lines 72–77) without the new required fields. This test expects validation to pass but will fail after this PR merges.
Remediation: Add "verified_variables": [] and "unchecked_variables": [] to the finding object.
[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/challenger.md:48 — The challenger sub-agent's adjudicated_findings output format template does not include the new required fields. The challenger reshapes findings and may drop these fields if the template doesn't show them.
Remediation: Add the fields to the adjudicated_findings template.

Low

[missing-struct-fields] internal/cli/postreview.go:159 — The Go ReviewFinding struct does not include the new fields. json.Unmarshal will silently discard them. No current logic uses these fields, so this is data loss only — not a runtime error.
[misleading-label] internal/scaffold/fullsend-repo/schemas/review-result.schema.json:56 — The PR body describes the fields as "optional" but the schema adds them to the required array. The implementation correctly matches issue Review agent: require structured enumeration of verified vs unchecked variables in sanitization findings #2095's requirement; the PR description is imprecise.

Info

[tier-classification] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — PR uses feat(#2095) prefix. Adding new required fields to an output schema is reasonably classified as a feature.
[test-adequacy] internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh:197 — Tests cover empty-string rejection for verified_variables but not unchecked_variables. Both arrays share identical JSON Schema constraints, so validation is identical.
[test-coverage-completeness] internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh — Test additions follow established patterns and are thorough.

Previous run (2)

Review

Findings

Medium

[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:380 — The standard finding format template (lines 380–389) and the sub-agent failure finding template (lines 405–412) do not include verified_variables or unchecked_variables. After this PR merges, the schema requires both fields on every finding (additionalProperties: false). Sub-agents and the orchestrator following these templates will produce findings that fail schema validation, triggering harness retry iterations.
Remediation: Add "verified_variables": [] and "unchecked_variables": [] to both finding templates.

Low

[misleading-label] internal/scaffold/fullsend-repo/schemas/review-result.schema.json:54 — The PR body describes the fields as "optional" but the schema adds them to the required array. The PR body also says "force," creating an internal contradiction. The implementation correctly matches issue Review agent: require structured enumeration of verified vs unchecked variables in sanitization findings #2095's requirement for mandatory structured enumeration; the description is imprecise rather than a scope violation.
[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/challenger.md:48 — The challenger sub-agent's output format template does not include verified_variables or unchecked_variables. Data is not lost in practice because the challenger passes through input findings and the orchestrator strips only challenger-specific fields.
[consumer-completeness] internal/cli/postreview.go:159 — The Go ReviewFinding struct does not include verified_variables or unchecked_variables fields. json.Unmarshal will silently discard them. No current logic uses these fields.
[incomplete-doc] internal/scaffold/fullsend-repo/skills/pr-review/meta-prompt.md:14 — The output format section in meta-prompt.md does not include verified_variables and unchecked_variables. This template is composed into every sub-agent's spawn prompt. Sub-agents that need these fields (security) already have explicit instructions in their own definitions.

Info

[test-adequacy] internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh:197 — Tests cover empty-string rejection for verified_variables but not unchecked_variables. Both arrays share identical JSON Schema constraints, so validation is identical.
[tier-classification] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — PR uses feat(#2095) prefix. Adding new required fields to an output schema is reasonably classified as a feature.
[architectural-coherence] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — Schema change aligns with ADR 0022 (harness-level output schema enforcement) and is additive/backward-compatible at the schema level.
[requirement-escalation] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/security.md:173 — Documentation now states both arrays "are required on every finding," matching the schema's required constraint. This resolves the previous discrepancy between documentation-level and schema-level requirements.

Previous run (3)

Review

Findings

Medium

[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:380 — The standard finding format template (lines 380–389) and the sub-agent failure finding template (lines 405–412) do not include verified_variables or unchecked_variables. After this PR merges, the schema requires both fields on every finding (additionalProperties: false). Sub-agents and the orchestrator following these templates will produce findings that fail schema validation, triggering harness retry iterations.
Remediation: Add "verified_variables": [] and "unchecked_variables": [] to both finding templates.

Low

[misleading-label] internal/scaffold/fullsend-repo/schemas/review-result.schema.json:54 — The PR body describes the fields as "optional" but the schema adds them to the required array. The PR body also says "force," creating an internal contradiction. The implementation correctly matches issue Review agent: require structured enumeration of verified vs unchecked variables in sanitization findings #2095's requirement for mandatory structured enumeration; the description is imprecise rather than a scope violation.
[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/challenger.md:48 — The challenger sub-agent's output format template does not include verified_variables or unchecked_variables. Data is not lost in practice because the challenger passes through input findings and the orchestrator strips only challenger-specific fields.
[consumer-completeness] internal/cli/postreview.go:159 — The Go ReviewFinding struct does not include verified_variables or unchecked_variables fields. json.Unmarshal will silently discard them. No current logic uses these fields.
[incomplete-doc] internal/scaffold/fullsend-repo/skills/pr-review/meta-prompt.md:14 — The output format section in meta-prompt.md does not include verified_variables and unchecked_variables. This template is composed into every sub-agent's spawn prompt. Sub-agents that need these fields (security) already have explicit instructions in their own definitions.

Info

[test-adequacy] internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh:197 — Tests cover empty-string rejection for verified_variables but not unchecked_variables. Both arrays share identical JSON Schema constraints, so validation is identical.
[tier-classification] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — PR uses feat(#2095) prefix. Adding new required fields to an output schema is reasonably classified as a feature.
[architectural-coherence] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — Schema change aligns with ADR 0022 (harness-level output schema enforcement) and is additive/backward-compatible at the schema level.
[requirement-escalation] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/security.md:173 — Documentation now states both arrays "are required on every finding," matching the schema's required constraint. This resolves the previous discrepancy between documentation-level and schema-level requirements.

Previous run (4)

Review

Findings

Low

[consumer-completeness] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/challenger.md:48 — The challenger sub-agent's output format template does not include verified_variables or unchecked_variables. The challenger passes through input findings and the orchestrator strips only challenger-specific fields, so data is not lost in practice. Updating the example would improve documentation completeness.
[incomplete-doc] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:378 — The finding object format example in SKILL.md step 5 omits verified_variables and unchecked_variables. Sub-agents that need these fields already have explicit instructions in their own skill files.
[incomplete-doc] internal/scaffold/fullsend-repo/skills/pr-review/meta-prompt.md:12 — The output format section in meta-prompt.md does not include verified_variables and unchecked_variables. Sub-agents that need these fields receive explicit instructions in security.md and code-review/SKILL.md.
[requirement-escalation] internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/security.md — Documentation states both arrays "must be present" in certain contexts while the schema marks them as optional. This is a deliberate pattern: schema-level optionality with documentation-level requirements for specific contexts.
[consumer-completeness] internal/cli/postreview.go:159 — The Go ReviewFinding struct does not include verified_variables or unchecked_variables fields. json.Unmarshal will silently discard them. No current logic uses these fields.

Info

[test-adequacy] internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh:173 — Tests cover empty-string rejection for verified_variables but not unchecked_variables. Both arrays share identical JSON Schema constraints, so validation is identical.
[tier-classification] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — PR uses feat(#2095) prefix. Adding new optional fields to an output schema is reasonably classified as a feature.
[documentation-consistency] internal/scaffold/fullsend-repo/agents/review.md:253 — New table rows use longer descriptions than existing rows because the fields require more context about conditional requirements.
[schema-evolution] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — Schema change is additive and backward-compatible.
[architectural-coherence] internal/scaffold/fullsend-repo/schemas/review-result.schema.json — Change aligns with ADR 0022 (harness-level output schema enforcement).

ralphbean

I think this needs a couple changes before we can merge. See inline.

ralphbean · 2026-06-16T21:06:39Z

          "description": "True when this non-blocking finding should be tracked as a follow-up issue if the review approves."
+        },
+        "verified_variables": {
+          "type": "array",


[medium] These fields aren't in the finding's required array, so an agent can omit them on a sanitization finding and pass validation. The prose in security.md and SKILL.md says "both arrays must be present" — but the schema doesn't enforce it, and category is free-text so there's no way to key a conditional required off "this is a sanitization finding."

I think the simplest fix is making both fields unconditionally required, with [] as the valid value for non-security findings. That way the agent always has to consciously decide — and the schema actually catches omissions instead of silently accepting them.

ralphbean · 2026-06-16T21:06:39Z

 | `description` | string  | yes      | Finding description (min 1 char)              |
 | `remediation` | string  | no       | Suggested fix                                 |
 | `actionable`  | boolean | no       | When true on low/info findings in an `approve` result, marks the finding for future follow-up issue creation (temporarily disabled; see #1137) |
+| `verified_variables` | array | no | Variables confirmed as having the security control applied. Required in findings that identify a sanitization or security control function. |


[moderate] The jq templates below (lines 289–298) are what the agent copy-pastes to build its output. They don't include verified_variables or unchecked_variables. If the fields become required, these templates produce invalid JSON that fails validation and burns a retry iteration.

Worth updating the request-changes/reject template to include them — and the findings array structure if there's an inline example of that too.

ralphbean · 2026-06-18T15:28:19Z

/fs-fix address those findings

fullsend-ai-coder · 2026-06-18T15:32:23Z

🤖 Finished Fix · ✅ Success · Started 3:32 PM UTC · Completed 3:40 PM UTC
Commit: 4e21a60 · View workflow run →

Make verified_variables and unchecked_variables unconditionally required in the finding schema so agents must always consciously populate them (using [] for non-security findings). Update jq templates in review.md to include the fields, preventing invalid JSON that would burn retry iterations. Add test for missing required variable arrays. Addresses review feedback on #2363

fullsend-ai-coder · 2026-06-18T15:40:09Z

🔧 Fix agent — iteration 1 (human-triggered)

Made verified_variables and unchecked_variables unconditionally required in the finding schema (with [] valid for non-security findings). Updated jq templates in review.md to include the new fields. Updated all test fixtures and documentation prose to match.

Fixed (4):

make verified/unchecked variables unconditionally required in schema (internal/scaffold/fullsend-repo/schemas/review-result.schema.json): Added verified_variables and unchecked_variables to the finding required array so agents must always populate them (using [] for non-security findings)
update jq templates to include verified/unchecked variable fields (internal/scaffold/fullsend-repo/agents/review.md): Updated the approve-with-findings and request-changes/reject jq templates to include verified_variables and unchecked_variables, and updated the finding table to mark both fields as required
update tests for new required fields (internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh): Added verified_variables/unchecked_variables to existing test fixtures that include findings, and added a new test case verifying that findings missing the required arrays are rejected
update prose in security.md and SKILL.md (internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/security.md): Updated security.md and SKILL.md prose to reflect that both arrays are unconditionally required on every finding, not just security findings

Tests: passed

_{Updated by fullsend fix agent}

fullsend-ai-review · 2026-06-18T15:44:07Z

🤖 Finished Review · ❌ Failure · Started 3:44 PM UTC · Completed 3:58 PM UTC
Commit: 455ec0c · View workflow run →

Make verified_variables and unchecked_variables unconditionally required in the finding schema so agents must always consciously populate them (using [] for non-security findings). Update jq templates in review.md to include the fields, preventing invalid JSON that would burn retry iterations. Add test for missing required variable arrays. Addresses review feedback on #2363

ben-alkov · 2026-06-18T19:03:48Z

/fs-review

fullsend-ai-review · 2026-06-18T19:08:13Z

🤖 Finished Review · ✅ Success · Started 7:08 PM UTC · Completed 7:22 PM UTC
Commit: 4e21a60 · View workflow run →

fullsend-ai-review

See the review comment for full details.

Make verified_variables and unchecked_variables unconditionally required in the finding schema so agents must always consciously populate them (using [] for non-security findings). Update jq templates in review.md to include the fields, preventing invalid JSON that would burn retry iterations. Add test for missing required variable arrays. Addresses review feedback on #2363

fullsend-ai-review · 2026-06-24T12:48:52Z

🤖 Finished Review · ✅ Success · Started 12:48 PM UTC · Completed 1:09 PM UTC
Commit: c7bbe6f · View workflow run →

fullsend-ai-review

See the review comment for full details.

fullsend-ai-review · 2026-06-24T13:09:21Z

    "finding": {
      "type": "object",
-      "required": ["severity", "category", "file", "description"],
+      "required": ["severity", "category", "file", "description", "verified_variables", "unchecked_variables"],


[critical] breaking-schema-change

The schema makes verified_variables and unchecked_variables required on every finding (additionalProperties: false), but multiple consumer templates and the Go struct have not been updated. (1) Three finding templates in SKILL.md (lines 380-389, 405-412, 537-544) omit the fields. (2) The meta-prompt.md template (lines 14-24) injected into every sub-agent system prompt omits them. (3) The challenger.md template (lines 48-59) omits them. (4) The review-result-label-actions-test.sh test at line 72 constructs a finding without the fields and expects validation to pass. (5) The Go ReviewFinding struct in internal/cli/postreview.go:166-174 lacks both fields. After this PR merges, all agents following any of these templates will produce findings that fail schema validation, triggering harness retry loops per ADR 0022.

Suggested fix: Either (a) make the two new fields optional (remove from required array) and rely on documentation to ensure security sub-agents populate them, or (b) update ALL consumers atomically in this PR: SKILL.md templates, meta-prompt.md, challenger.md, the label-actions test, and the Go struct.

/fs-fix Address this finding by updating ALL consumers atomically in this PR: SKILL.md templates, meta-prompt.md, challenger.md, the label-actions test, and the Go struct.

Make verified_variables and unchecked_variables unconditionally required in the finding schema so agents must always consciously populate them (using [] for non-security findings). Update jq templates in review.md to include the fields, preventing invalid JSON that would burn retry iterations. Add test for missing required variable arrays. Addresses review feedback on #2363

…s schema Add optional `verified_variables` and `unchecked_variables` array fields to the finding object in review-result.schema.json. These fields force the review agent's security sub-agent to structurally enumerate which variables it verified as sanitized and which it did not, preventing blanket safety claims from partial verification. Changes: - review-result.schema.json: add both arrays (string items, minLength 1) to the finding definition under additionalProperties: false - security.md (sub-agent): update verification methodology step 3 to require populating the new structured fields instead of prose - code-review/SKILL.md: update exhaustive security-control verification guidance to reference the new fields - agents/review.md: add new fields to the finding object documentation table - validate-output-schema-test.sh: add three test cases (partial coverage valid, full coverage valid, empty-string variable rejected) and update the allowed-properties assertion Pre-commit could not run (shellcheck install failed due to network restrictions in the sandbox). The post-script runs pre-commit authoritatively. Closes #2095

Make verified_variables and unchecked_variables unconditionally required in the finding schema so agents must always consciously populate them (using [] for non-security findings). Update jq templates in review.md to include the fields, preventing invalid JSON that would burn retry iterations. Add test for missing required variable arrays. Addresses review feedback on #2363

ben-alkov · 2026-06-29T20:08:56Z

/fs-fix Address #2363 (comment) by updating ALL consumers atomically in this PR: SKILL.md templates, meta-prompt.md, challenger.md, the label-actions test, and the Go struct.

fullsend-ai-coder · 2026-06-29T20:12:01Z

🤖 Finished Fix · ✅ Success · Started 8:12 PM UTC · Completed 8:20 PM UTC
Commit: 104508d · View workflow run →

Update all consumers of the review finding schema to include the now- required verified_variables and unchecked_variables arrays: - SKILL.md: three finding templates (standard, sub-agent-failure, challenger-failure) - meta-prompt.md: sub-agent output format template - challenger.md: adjudicated_findings template - review-result-label-actions-test.sh: request-changes test finding - postreview.go: ReviewFinding Go struct Addresses review feedback on #2363

fullsend-ai-coder · 2026-06-29T20:20:30Z

🔧 Fix agent — iteration 2 (human-triggered)

Updated all 5 consumer locations to include the now-required verified_variables and unchecked_variables arrays: 3 SKILL.md templates, meta-prompt.md, challenger.md, label-actions test, and Go struct. All schema validation tests pass. Go code compiles and vets cleanly.

Fixed (5):

SKILL.md finding templates missing verified/unchecked variables (internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md): Added verified_variables and unchecked_variables arrays to all three finding JSON templates (standard finding, sub-agent-failure, challenger-failure)
meta-prompt.md template missing verified/unchecked variables (internal/scaffold/fullsend-repo/skills/pr-review/meta-prompt.md): Added verified_variables and unchecked_variables arrays to the sub-agent output format template
challenger.md template missing verified/unchecked variables (internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/challenger.md): Added verified_variables and unchecked_variables arrays to the adjudicated_findings template
label-actions test finding missing required fields (internal/scaffold/fullsend-repo/schemas/review-result-label-actions-test.sh): Added verified_variables and unchecked_variables arrays to the request-changes test finding so it passes schema validation
Go ReviewFinding struct missing verified/unchecked variable fields (internal/cli/postreview.go): Added VerifiedVariables and UncheckedVariables []string fields to the ReviewFinding struct to match the schema

Tests: passed

_{Updated by fullsend fix agent}

ralphbean

This is an automated comment. We are moving agent content from internal/scaffold/fullsend-repo/ to https://github.com/fullsend-ai/agents -- changes should be made to agent definitions there going forwards.

ben-alkov · 2026-07-15T16:12:36Z

Will move to agents repo

ben-alkov · 2026-07-24T21:28:21Z

Superseded by fullsend-ai/agents#446

github-actions Bot deployed to site-preview June 16, 2026 20:05 View deployment

fullsend-ai-review Bot approved these changes Jun 16, 2026

View reviewed changes

Comment thread internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh

Comment thread internal/scaffold/fullsend-repo/agents/review.md Outdated

fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Jun 16, 2026

ralphbean requested changes Jun 16, 2026

View reviewed changes

github-actions Bot deployed to site-preview June 18, 2026 15:41 View deployment

ben-alkov self-assigned this Jun 18, 2026

ben-alkov added agent/review Review agent ok-to-test Allow e2e CI to run after maintainer review (must be re-applied after each push) labels Jun 18, 2026

ben-alkov force-pushed the agent/2095-structured-variable-enumeration branch from 455ec0c to f801081 Compare June 18, 2026 19:03

github-actions Bot removed the ok-to-test Allow e2e CI to run after maintainer review (must be re-applied after each push) label Jun 18, 2026

github-actions Bot deployed to site-preview June 18, 2026 19:04 View deployment

fullsend-ai-review Bot suggested changes Jun 18, 2026

View reviewed changes

Comment thread internal/scaffold/fullsend-repo/schemas/review-result.schema.json

Comment thread internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh

fullsend-ai-review Bot removed the ready-for-merge All reviewers approved — ready to merge label Jun 18, 2026

ben-alkov force-pushed the agent/2095-structured-variable-enumeration branch from f801081 to c7bbe6f Compare June 24, 2026 12:45

github-actions Bot deployed to site-preview June 24, 2026 12:46 View deployment

fullsend-ai-review Bot suggested changes Jun 24, 2026

View reviewed changes

fullsend-ai-review Bot added the security Security threat model and related concerns label Jun 24, 2026

ben-alkov force-pushed the agent/2095-structured-variable-enumeration branch from c7bbe6f to de3a0d9 Compare June 29, 2026 19:10

github-actions Bot deployed to site-preview June 29, 2026 19:12 View deployment

fullsend-ai-coder Bot added 2 commits June 29, 2026 16:07

ben-alkov force-pushed the agent/2095-structured-variable-enumeration branch from de3a0d9 to 6f971b7 Compare June 29, 2026 20:07

github-actions Bot deployed to site-preview June 29, 2026 20:09 View deployment

github-actions Bot deployed to site-preview June 29, 2026 20:22 View deployment

ralphbean requested changes Jul 6, 2026

View reviewed changes

ben-alkov closed this Jul 15, 2026

ben-alkov mentioned this pull request Jul 24, 2026

feat(#2095): add verified/unchecked variable fields to review findings schema fullsend-ai/agents#446

Open

4 tasks

github-actions Bot deleted the agent/2095-structured-variable-enumeration branch August 2, 2026 04:08

Uh oh!

Conversation

fullsend-ai-coder Bot commented Jun 16, 2026

Post-script verification

Uh oh!

github-actions Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E tests did not run

Uh oh!

github-actions Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Site preview

Uh oh!

fullsend-ai-review Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jun 16, 2026

Codecov Report

Uh oh!

fullsend-ai-review Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review

Findings

Critical

Review

Findings

Critical

High

Medium

Review

Findings

Medium

Review

Findings

Low

Review

Findings

Critical

High

Medium

Low

Info

Review

Findings

Medium

Low

Info

Review

Findings

Medium

Low

Info

Review

Findings

Low

Info

Uh oh!

Uh oh!

Uh oh!

ralphbean left a comment

Choose a reason for hiding this comment

Uh oh!

ralphbean Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

ralphbean Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

ralphbean commented Jun 18, 2026

Uh oh!

fullsend-ai-coder Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fullsend-ai-coder Bot commented Jun 18, 2026

🔧 Fix agent — iteration 1 (human-triggered)

Uh oh!

fullsend-ai-review Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ben-alkov commented Jun 18, 2026

github-actions Bot commented Jun 16, 2026 •

edited

Loading

github-actions Bot commented Jun 16, 2026 •

edited

Loading

fullsend-ai-review Bot commented Jun 16, 2026 •

edited

Loading

fullsend-ai-review Bot commented Jun 16, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Jun 18, 2026 •

edited

Loading

fullsend-ai-review Bot commented Jun 18, 2026 •

edited

Loading

fullsend-ai-review Bot commented Jun 18, 2026 •

edited

Loading

fullsend-ai-review Bot commented Jun 24, 2026 •

edited

Loading

fullsend-ai-coder Bot commented Jun 29, 2026 •

edited

Loading