Repository navigation
Conversation
Adds examples/testsuite-security-tests: a Portman config that turns the security rules in an OpenAPI document into tests, mapped to the OWASP API Security Top 10 (2023), using only existing features: - strict schema validation (undeclared response fields, API3) - variations without / with forged credentials asserted with extendTests, so they run even when no 401 is documented (API2) - fuzzing that must yield a 4xx, not a 5xx or 2xx (API4/API8) - security header and stack disclosure checks (API8) Fix: schemaValidation.additionalProperties = false also closed free-form objects, maps (additionalProperties: schema) and allOf members, so valid responses failed. These now stay open; explicit `additionalProperties: true` is still overridden as before. On the Apideck Connector API this cut strict-mode errors from 501 to 269, all of them real undeclared fields. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related changes.
1.
examples/testsuite-security-testsA Portman config that turns the security rules already in an OpenAPI document into tests, mapped to the OWASP API Security Top 10 (2023). It only uses existing features, so users can copy it into their own config:
schemaValidationwithadditionalProperties: falseoverwriteRequestSecurity.remove+extendTestsextendTestsextendTestsfuzzingvariation +extendTestsX-Content-Type-Options: nosniffcontentTests.responseHeaderTestsX-Powered-By, no version inServer, no*CORS with credentialsextendTestsThe auth variations assert the status with
extendTestsinstead ofopenApiResponse: "401", because a variation targeting a response code is skipped when the operation doesn't document it. That way an endpoint that forgets its auth middleware is caught even when its spec lists no 401.The readme is explicit about scope: no attack payloads (pair with a DAST tool), only spec-listed endpoints, and object-level authorization (API1/API5) needs variations the user writes with per-tenant test users.
The example is added to
ExamplesNormalizedSnapshots, and the README feature list links to it.2. Fix: strict schema mode closed schemas that can't be closed
schemaValidation.additionalProperties: falsesetadditionalProperties: falseon everytype: objectnode. That broke valid responses for:type: objectwithoutproperties): every key was rejectedadditionalProperties: { type: string }): the value schema was replaced byfalseallOfmembers: each member only declares part of the properties, so the composed object always failedThese now stay open. Objects that declare
propertiesare closed as before, including the existing behaviour of overriding an explicitadditionalProperties: true(the existing snapshot test is unchanged). Nested objects insideallOfmembers are still closed.On the Apideck Connector API (live run, sandbox key) strict mode went from 501 errors to 269, and every remaining one is a real undeclared field.
Test plan
allOfmembers open, still closes objects nested inallOfmembersnpm run lint:cinoauth, invalid-credentials variation sends the forged bearer token, fuzzing generates 4xx-asserting variationsIndependent of #762 (circular
$refvalidation); together they let the preset run on specs with recursive schemas.🤖 Generated with Claude Code