Skip to content

feat: security regression test example (OWASP API Top 10); fix strict schema mode - #763

Open
Gdewilde wants to merge 1 commit into
mainfrom
feat/security-preset
Open

Gdewilde wants to merge 1 commit into
mainfrom
feat/security-preset

Conversation

@Gdewilde

@Gdewilde Gdewilde commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Two related changes.

1. examples/testsuite-security-tests

A Portman config that turns the security rules already in an OpenAPI document into tests, mapped to the OWASP API Security Top 10 (2023). It only uses existing features, so users can copy it into their own config:

Check How OWASP
No undeclared response fields schemaValidation with additionalProperties: false API3
Rejected without credentials (401/403) variation with overwriteRequestSecurity.remove + extendTests API2
Rejected with invalid credentials variation with a forged bearer token + extendTests API2
No stack traces in error bodies extendTests API3/API8
Out-of-spec input gets a 4xx, not 5xx/2xx fuzzing variation + extendTests API4/API8
HSTS, X-Content-Type-Options: nosniff contentTests.responseHeaderTests API8
No X-Powered-By, no version in Server, no * CORS with credentials extendTests API8

The auth variations assert the status with extendTests instead of openApiResponse: "401", because a variation targeting a response code is skipped when the operation doesn't document it. That way an endpoint that forgets its auth middleware is caught even when its spec lists no 401.

The readme is explicit about scope: no attack payloads (pair with a DAST tool), only spec-listed endpoints, and object-level authorization (API1/API5) needs variations the user writes with per-tenant test users.

The example is added to ExamplesNormalizedSnapshots, and the README feature list links to it.

2. Fix: strict schema mode closed schemas that can't be closed

schemaValidation.additionalProperties: false set additionalProperties: false on every type: object node. That broke valid responses for:

  • free-form objects (type: object without properties): every key was rejected
  • maps (additionalProperties: { type: string }): the value schema was replaced by false
  • allOf members: each member only declares part of the properties, so the composed object always failed

These now stay open. Objects that declare properties are closed as before, including the existing behaviour of overriding an explicit additionalProperties: true (the existing snapshot test is unchanged). Nested objects inside allOf members are still closed.

On the Apideck Connector API (live run, sandbox key) strict mode went from 501 errors to 269, and every remaining one is a real undeclared field.

Test plan

  • Full jest suite: 70 suites, 778 passed, 14 skipped; all existing snapshots unchanged, 1 new example snapshot
  • New Ajv-backed tests: closes declared objects, leaves free-form objects open, keeps map value schemas, leaves allOf members open, still closes objects nested in allOf members
  • npm run lint:ci
  • Generated example collection: no-credentials variation sends noauth, invalid-credentials variation sends the forged bearer token, fuzzing generates 4xx-asserting variations
  • Live run of the preset against the Apideck Connector API (10 GET operations): auth variations, fuzzing, header and schema checks all execute as intended

Independent of #762 (circular $ref validation); together they let the preset run on specs with recursive schemas.

🤖 Generated with Claude Code

Adds examples/testsuite-security-tests: a Portman config that turns the
security rules in an OpenAPI document into tests, mapped to the OWASP API
Security Top 10 (2023), using only existing features:
- strict schema validation (undeclared response fields, API3)
- variations without / with forged credentials asserted with extendTests,
  so they run even when no 401 is documented (API2)
- fuzzing that must yield a 4xx, not a 5xx or 2xx (API4/API8)
- security header and stack disclosure checks (API8)

Fix: schemaValidation.additionalProperties = false also closed free-form
objects, maps (additionalProperties: schema) and allOf members, so valid
responses failed. These now stay open; explicit `additionalProperties:
true` is still overridden as before. On the Apideck Connector API this
cut strict-mode errors from 501 to 269, all of them real undeclared
fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant