Skip to content

Reduce validation cost and add performance regression coverage - #354

Open
satyakigh wants to merge 2 commits into
mainfrom
performance-fixes
Open

Reduce validation cost and add performance regression coverage#354
satyakigh wants to merge 2 commits into
mainfrom
performance-fixes

Conversation

@satyakigh

@satyakigh satyakigh commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

This change reduces cold and repeated CloudFormation validation cost while preserving report output and engine parity.

Shared validation data

  • Uses one typed schema metadata catalog across the schema validator and both rule engines.
  • Shares the schema validator and metadata through reference-counted state instead of rebuilding equivalent catalogs.
  • Reuses memoized resolved scenarios through shared immutable allocations during composition evaluation.
  • Preserves additional-schema overlay behavior while replacing only affected metadata entries.

Schema validation

  • Validates authored properties and reachable conditional-branch properties instead of traversing every absent property in a provider schema.
  • Reuses the model's resource-type index rather than rebuilding a type set from every resource.
  • Preserves required-property, unknown-property, nested composition, and conditional-property behavior.

Rule evaluation

  • Groups primary-identifier scenarios by comparison identity before compatibility work while preserving deterministic comparison order.
  • Restricts primary-identifier and regional-enum processing to resource types present in the template.
  • Replaces quadratic VPC-to-subnet association scans with indexed VPC CIDRs and one subnet pass.
  • Uses membership-oriented regional enum sets without changing ordered compatibility APIs or diagnostic text.

Performance regression coverage

  • Adds an exact base/head performance gate for initialization, first validation, repeated validation, phase timings, peak RSS, and report equality.
  • Runs every committed YAML/JSON security fixture through both engines.
  • Adds a 500-resource dense fan-out fixture with 11,875 resource-to-resource references.
  • Keeps parser-only lifecycle ratios visible while gating their parse and validation time separately from fixed engine startup.

Performance results

Representative results versus main:

Workload CEL warm total Rego warm total Init + first Peak RSS
500 unique resources 77% faster 25% faster 20–33% faster 11–15% lower
500 duplicate resources 63% faster 13% faster 15–32% faster 10–16% lower
100 conditional resources 67% faster 3% faster 14–18% faster 10–12% lower
Mixed real templates 36% faster 8% faster 14–16% faster 10–11% lower
500 independent resources 62% faster 14% faster 12–23% faster 9–11% lower
Scenario-assignment boundary 40% faster 37% faster 31–32% faster 9–10% lower

One-time initialization moves more work into engine construction, while initialization plus first validation is lower across these workloads.

@satyakigh satyakigh changed the title Optimize repeated template validation Reduce repeated CloudFormation validation cost Aug 28, 2026
@satyakigh satyakigh changed the title Reduce repeated CloudFormation validation cost Reduce validation cost and add performance regression coverage Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant