TRAM (Test Runner for Assertion Manifests) is a framework for creating executable behavioral models of HTTP APIs. It validates those models and gathers evidence from running APIs to verify that observable behavior matches the intended design.
|
Rather than focusing on implementation details, TRAM focuses on what can be observed at the API surface: the resources, actions, workflows, and rules that define how a system behaves. Assertions are organized into progressively richer layers, moving from endpoint availability and response structure to business behavior, workflows, and governance constraints. This allows teams to express operational intent as a durable behavioral model that remains valuable even as implementations evolve. |
|
-
Node.js API guide — load, validate, and execute manifests programmatically
-
Node.js Tasks example — run the existing Tasks manifest from Node.js
TRAM combines:
- a manifest-driven test format
- a reusable assertion engine
- a portable HTTP test runner
- stable runtime interpolation support
- native type assertions
- optional property assertions
- object-map and collection assertions
- layered behavioral modeling
- workflow-oriented behavioral validation
- an AI Coaching workflow focused on learning and augmentation rather than pure automation
Manifest version 0.2 introduces response capture.
Capture allows a test to extract values from an HTTP response (such as resource identifiers or hypermedia links) and reuse those values in subsequent requests. This makes it possible to write behavioral tests for APIs that generate identifiers dynamically or expose navigational affordances.
Earlier (0.1) manifests remain supported and continue to execute without modification.
A TRAM run can produce several complementary artifacts.
| Artifact | Purpose |
|---|---|
| Manifest | Defines the expected API behavior. |
| HTTP Transcript | Records the observed HTTP request/response conversation. |
| Report | Evaluates the observed behavior against the manifest. |
| Evidence | The complete collection of artifacts from a test run. |
The transcript and report serve different purposes.
The HTTP Transcript records what happened during execution. The Report evaluates whether the observed behavior satisfied the behavioral expectations expressed in the manifest.
{
"name": "Smallest TRAM manifest",
"config": {
"baseUrl": "http://localhost:3000"
},
"tests": [
{
"name": "GET /tasks returns 200",
"method": "GET",
"path": "/tasks",
"expect": {
"status": 200
}
}
]
}This is the smallest useful complete TRAM manifest:
- one manifest
- one test
- one request
- one behavioral assertion
The capture property records values observed in an HTTP response and makes them available to later requests.
{
"id": "task-create",
"method": "POST",
"path": "/tasks",
"bodyType": "json",
"body": "$data.task.capture.valid",
"expect": {
"status": 201
},
"capture": {
"createdTaskId": "body.id"
}
}Later tests can reference the captured value:
{
"id": "task-get",
"method": "GET",
"path": "/tasks/${capture.createdTaskId}",
"expect": {
"status": 200
}
}Capture works with response bodies, headers, and other observable response values. See the Manifest Specification for the complete syntax.
TRAM explores a narrow problem:
How do we make behavioral expectations directly visible, portable, executable, and reviewable?
The core artifact is the manifest:
The manifest defines:
- requests
- request bodies
- assertions
- expected behaviors
- shared test data
- runtime interpolation values
Assertions become directly inspectable operational statements.
Simple behavioral assertion:
{
"path": "$.status",
"equals": "active"
}Meaning:
The resource status must be "active".
Optional property assertion for evolving representations:
{
"path": "$",
"each": {
"property": "description",
"optional": true,
"type": "string"
}
}Meaning:
"description" may be absent.
If present, it must be a string.
Hypermedia affordance assertion:
{
"path": "$._links",
"eachProperty": {
"hasProperties": ["href", "method"]
}
}Meaning:
Every affordance must define both a target URL and an HTTP method.
Collection behavioral assertion:
{
"path": "$",
"each": {
"property": "status",
"oneOf": ["active", "pending", "completed"]
}
}Meaning:
Every returned resource must have a recognized workflow state.
Nested affordance traversal assertion:
{
"path": "$",
"each": {
"path": "$._links",
"eachProperty": {
"hasProperties": ["href", "method"]
}
}
}Meaning:
Every returned resource must expose affordances
that define both a target URL and an HTTP method.
TRAM supports partial and evolving representations while preserving explicit behavioral validation.
TRAM organizes behavioral testing into six progressive layers.
| Level | Focus | Question |
|---|---|---|
| 0 | Surface | Can the API be reached? |
| 1 | Shape | Do resources and affordances appear correctly? |
| 2 | Safe behavior | Do navigation, lookup, filtering, and query interactions behave correctly? |
| 3 | Unsafe behavior | Do isolated state-changing actions behave correctly? |
| 4 | Workflow | Can meaningful operational narratives be completed successfully? |
| 5 | Governance | Are policies, constraints, and semantic rules enforced correctly? |
See Behavioral Modeling for APIs for the six-layer model.
The layers are additive rather than replacement-oriented. Each layer narrows debugging scope while preserving readable behavioral intent.
TRAM is designed around several principles:
- behavioral tests over implementation tests
- portable manifests over framework lock-in
- readable intent over clever abstractions
- explicitness over hidden runtime behavior
- low-noise reporting
- augmentation and learning over one-shot generation
The long-term direction is an AI Coach that helps users learn behavioral API testing while collaboratively constructing executable manifests.
Current implementation includes:
- manifest specification (
api-tests.json) - dependency-free assertion engine
- dependency-free HTTP runner
- body/header/status assertions
- collection assertions (
each) - object-map assertions (
eachProperty) - native type assertions (
type) - optional property assertions (
optional) - range assertions (
range) - stable run-scoped variables
- runtime interpolation (
${data.*}) - object injection (
$data.*) - capture values from responses and reuse them in later requests
- happy-path and sad-path testing
- JSON, form, and text request body support
- workflow-oriented behavioral modeling
- machine-readable reporting
- real API validation against a sample CRUD-style task API
- HTTP transcript generation
.
├── README.md
├── package.json
├── api-tests.json
├── bin/
│ └── tram
├── lib/
│ └── assertions.js
├── docs/
└── sample-api/
The CLI accepts a manifest filename followed by options; validate and run are not separate subcommands.
tram <manifest-file> [options]Options:
-v, --verbose Print passing assertion details
-r, --report <file> Write behavioral report (JSON)
-t, --transcript <file> Write HTTP transcript
-c, --validate Validate the manifest without making HTTP requests
-h, --help Show help
TRAM 0.1.0 beta is distributed through npm as @mamund/tram using the beta distribution tag. The following installation commands apply after publication.
TRAM requires Node.js 18 or later and has no runtime dependencies.
Install the command globally:
npm install --global @mamund/tram@beta
tram --helpAlternatively, add TRAM to an existing Node.js project:
npm install --save-dev @mamund/tram@beta
npx tram --helpValidate a manifest without sending HTTP requests:
tram api-tests.json --validateExecute a manifest and save the resulting evidence:
tram api-tests.json --report results.json --transcript transcript.httpThe manifest is supplied by your project; the npm package does not install a sample API or sample manifest. Start with the public API quick start. The repository also contains a local Tasks API example for more advanced testing.
To work on TRAM itself, clone the source repository:
git clone https://github.com/mamund/tram.git
cd tram
npm test
npm linkThe npm link command is for developing TRAM from source. It is not required for npm consumers.
Tests are defined declaratively in a manifest:
{
"name": "Create task",
"method": "POST",
"path": "/tasks/${data.stableId}",
"body": "$data.task.valid",
"expect": {
"status": 201,
"body": [
{
"path": "$.status",
"equals": "active"
}
]
}
}The manifest acts as both:
- executable configuration
- behavioral operational artifact
The data section stores reusable request and runtime values.
Example:
{
"data": {
"stableId": "${randomId}"
}
}The generated value remains stable throughout the current test run.
Later requests can reference the same value:
{
"path": "/tasks/${data.stableId}"
}TRAM also supports response capture.
Values observed in one response may be reused later in the same test run.
Example:
"capture": {
"taskId": "body.id"
}Later requests can reference the captured value:
"path": "/tasks/${capture.taskId}"This enables coordinated multi-step behavioral flows without introducing custom scripting.
Use:
"$data.someObject"when injecting structured runtime objects.
Use:
"${data.someValue}"when interpolating values inside strings.
Examples:
Correct object injection:
"body": "$data.createTask"Correct string interpolation:
"path": "/tasks/${data.knownTaskId}"Captured response values use:
"${capture.taskId}"These values are populated during test execution from earlier HTTP responses.
The repository includes a dedicated capture example:
See the capture examples in the [repository](https://github.com/mamund/tram).
The example demonstrates:
- creating a resource
- capturing values from the response
- reusing those values in subsequent requests
- following captured hypermedia links
The assertion library currently supports:
exists
equals
contains
oneOf
type
range
isArray
hasProperties
length
minLength (deprecated)
each
eachProperty
Native type assertions support:
string
number
boolean
array
object
null
Example native type assertion:
{
"path": "$.priority",
"type": "number"
}Example optional property assertion:
{
"path": "$",
"each": {
"property": "description",
"optional": true,
"type": "string"
}
}This assertion means:
"description" may be absent
if present, it must still validate as a string
TRAM distinguishes between arrays and object maps.
Use:
eachfor arrayseachPropertyfor object maps
Examples:
[
{...},
{...}
]=> each
{
"self": {...},
"edit": {...}
}=> eachProperty
TRAM also distinguishes between:
pathfor structural traversalpropertyfor scalar leaf checks
Example structural traversal:
{
"path": "$",
"each": {
"path": "$._links",
"eachProperty": {
"hasProperties": ["href", "method"]
}
}
}Example scalar leaf assertion:
{
"path": "$",
"each": {
"property": "status",
"equals": "active"
}
}The assertion model supports:
- collection traversal
- nested traversal
- object-map iteration
- native value validation
- optional property validation
- hypermedia affordance validation
while remaining declarative and inspectable.
TRAM intentionally limits type assertions to native value categories.
The following are currently out of scope:
uuid
email
uri
date-time
schema validation
TRAM manifests can model operational workflows rather than isolated endpoint checks.
TRAM models workflows through declarative sequencing rather than embedded scripting.
A workflow manifest may:
- create resources
- retrieve intermediate state
- apply mutations
- verify accumulated final state
This allows manifests to function as executable operational narratives.
Example workflow sequence:
create
read after create
edit
update status
assign user
set due date
read final accumulated state
Header assertions use:
{
"name": "content-type",
"contains": "application/json"
}Do not use path for header assertions.
TRAM supports multiple request body encodings:
json
form
text
Example:
{
"method": "PUT",
"path": "/tasks/task-1/status",
"bodyType": "form",
"body": "$data.task.updateStatus"
}Start the sample API:
node sample-api/index.jsRun the test suite:
tram api-tests.jsonVerbose mode:
tram api-tests.json --verboseGenerate an HTTP transcript:
tram api-tests.json --transcript transcript.httpGenerate a machine-readable report:
tram api-tests.json --report report.jsonGenerate both artifacts:
tram api-tests.json \
--report report.json \
--transcript transcript.http- running the sample project
- inspecting manifests
- understanding assertions
- understanding runtime interpolation
- exploring behavioral API testing workflows
Authoritative executable manifest model.
Defines:
- manifest structure
- request configuration
- assertion syntax
- optional property assertions
- traversal behavior
- runtime interpolation
- stable run-scoped variables
- collection assertions
- object-map assertions
- native type assertions
- body handling
Architectural discussion of:
- behavioral assertions
- operational artifacts
- hypermedia-oriented testing
- generated systems
- workflow-oriented behavioral modeling
- AI-assisted workflows
TRAM validates manifests before executing HTTP requests. Validation may also be invoked directly from the command line using the --validate option.
Validation currently includes:
- manifest file existence
- manifest JSON parsing
- top-level manifest structure
- required test fields
- supported HTTP methods
- supported request body types
- duplicate test IDs
- capture declarations
- capture path syntax
- capture identifier syntax
Example:
tram api-tests.json --validateIf the manifest is valid, TRAM reports success and exits without executing any requests. If validation fails, TRAM reports the validation errors and exits with a non-zero status.
Invalid manifests fail before execution begins.
TRAM reports multiple manifest validation problems in a single pass when possible.
TRAM distinguishes between:
- manifest authoring failures
- request/runtime failures
- behavioral assertion failures
TRAM can produce several complementary views of a test run.
- concise console output
- HTTP transcript of the observed conversation
- machine-readable behavioral report
The transcript records the observed HTTP conversation.
The report evaluates that conversation against the behavioral expectations expressed in the manifest.
Manifest validation is treated as a first-class operation, allowing behavioral models to be reviewed independently of execution.
TRAM is intentionally conservative.
Current releases avoid:
- framework dependencies
- custom scripting
- setup/teardown orchestration
- schema engines
- plugin systems
- hidden runtime behavior
The current emphasis is:
- clarity
- predictability
- behavior visibility
- manifest ergonomics
- reviewability
The AI Coaching direction includes:
- layered manifest generation
- traversal-aware assertion guidance
- workflow modeling support
- governance distinction guidance
- collaborative review cycles
- behavioral decomposition assistance
The eventual AI Coach layer will:
- inspect
server.jsand/or API Story documents - identify API behaviors
- propose candidate tests
- distinguish happy and sad paths
- review assertions collaboratively
- generate plausible first-pass manifests
The goal is not automatic test generation alone.
The goal is helping users understand behavioral API testing while collaboratively constructing executable manifests.
Typical TRAM progression:
Level 0 — endpoint availability
Level 1 — representation structure
Level 2 — lookup and filtering behavior
Level 3 — isolated mutation behavior
Level 4 — workflow continuity
Level 5 — governance and constraints
TRAM draws inspiration from:
- behavioral testing
- executable specifications
- hypermedia-oriented design
- affordance-centric APIs
- augmentation-oriented AI systems
- coaching-based human/machine collaboration
Early experimental project.
Interfaces and manifest formats will evolve during v0.x development.
Project repository:
https://github.com/mamund/tram