diff --git a/CHANGELOG.md b/CHANGELOG.md index e45332e..8a4a1d9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,31 @@ All notable changes to the CacheKit Protocol Specification. ## [Unreleased] +### Interop mode — `ns` and `nsapi` are reserved namespaces (LAB-5876) + +- [`spec/interop-mode.md` → Segment grammar](spec/interop-mode.md#segment-grammar): + `namespace` MUST NOT be `ns` or `nsapi`. The segment pattern admitted both, but the + resulting key starts `ns:` / `nsapi:`, which the server parses as namespace-prefixed + ([cache-key-format.md → Server-Side Requirements](spec/cache-key-format.md#server-side-requirements)): + rejected when the operation contains `.`, otherwise scoped to a namespace named after + the operation. The reservation is exact-match and namespace-only; `ns` and `nsapi` + stay valid operations. SDKs reject a reserved namespace at decoration / registration + time, on every backend. **Breaking for any deployment that uses namespace `ns` or + `nsapi`, on any backend:** it now raises at startup; migrate by renaming the namespace + (a full cache miss for that namespace). +- [`test-vectors/interop-mode.json`](test-vectors/interop-mode.json) 1.1.0: two error + vectors (`reject_reserved_namespace_ns`, `reject_reserved_namespace_nsapi`) and one key + vector (`reservation_scope`: namespace `nsapix`, operation `nsapi`) that pins the + reservation as namespace-only and exact-match. Counts: 34 key, 11 error. +- SaaS Considerations no longer calls the grammar a strict subset of what the server + accepts: the grammar admits `..` inside a segment, which the server rejects. +- SDK feature matrix: the "Test vectors in CI" cells note that fixture 1.1.0 is not yet + in any released SDK, linking the SDK PRs that vendor it. + `tools/interop-reference.py` builds every key vector through its validating + `interop_key`; `tools/interop-crosscheck.mjs` checks the segment grammar on key vectors + as well as error vectors, with the reserved names hard-coded rather than read from the + fixture. + ### Wire format — vendored-fixture coverage note corrected (LAB-1750) - [`spec/wire-format.md`](spec/wire-format.md) no longer says `cachekit-core` vendors diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 19d25f6..5e39aec 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -298,7 +298,7 @@ its spec: | Encryption (AES-256-GCM) | ✅ Compliant | ✅ Canonical (cachekit-core) | ✅ Compliant | ⚠️ Untested | | AAD v0x03 | ✅ Compliant (5 components — every auto serializer appends `original_type`; interop mode is the sole 4-component path) | ✅ Compliant (4 components) | ✅ Compliant (4 components) | ❌ Not implemented | | SaaS API | ✅ Compliant | ✅ Compliant (CachekitIO backend) | ✅ Compliant | ❌ Not implemented | -| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors | ⚠️ Pending | +| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) — fixture 1.1.0 (`ns`/`nsapi` namespace reservation) in [cachekit-py#350](https://github.com/cachekit-io/cachekit-py/pull/350), unreleased | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) — fixture 1.1.0 in [cachekit-rs#89](https://github.com/cachekit-io/cachekit-rs/pull/89), unreleased | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors — fixture 1.1.0 in [cachekit-ts#143](https://github.com/cachekit-io/cachekit-ts/pull/143), unreleased | ⚠️ Pending | | Interop mode ([spec](spec/interop-mode.md), opt-in) | ✅ Released — PyPI 0.14.0+¹⁷ ([#220](https://github.com/cachekit-io/cachekit-py/pull/220)) | ✅ Released — crates.io 0.4.0+ ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)) | ✅ Released — npm 0.1.3+ ([#71](https://github.com/cachekit-io/cachekit-ts/pull/71)) | ❌ Not implemented | > [!NOTE] diff --git a/spec/interop-mode.md b/spec/interop-mode.md index aca6127..48e4365 100644 --- a/spec/interop-mode.md +++ b/spec/interop-mode.md @@ -13,7 +13,8 @@ > each registry or the [SDK feature matrix](../sdk-feature-matrix.md#compliance-status) for current versions. > Server-side: the CachekitIO validator accepts interop-format keys > (`{namespace}:{operation}:{args_hash}` scopes to the `default` namespace; -> see [cache-key-format.md → Server-Side Requirements](cache-key-format.md#server-side-requirements)). +> see [cache-key-format.md → Server-Side Requirements](cache-key-format.md#server-side-requirements)), +> except a key with `..` in a segment ([SaaS Considerations](#saas-considerations)). > Design discussion: [Issue #1](https://github.com/cachekit-io/protocol/issues/1) · > Test vectors: [`test-vectors/interop-mode.json`](../test-vectors/interop-mode.json) · > Reference implementation: [`tools/interop-reference.py`](../tools/interop-reference.py) @@ -112,6 +113,17 @@ Lowercase ASCII letters, digits, `.`, `_`, `-`; 1–64 characters; must start wi letter or digit. SDKs MUST reject non-conforming segments with an error at decoration / registration time — never silently normalize. +`namespace` additionally MUST NOT be `ns` or `nsapi`: the CachekitIO server parses a key +starting `ns:` or `nsapi:` as namespace-prefixed +([cache-key-format.md → Server-Side Requirements](cache-key-format.md#server-side-requirements)), +so an interop key in either namespace would be rejected or scoped to a namespace named +after the operation. SDKs reject a reserved namespace like any other non-conforming +segment, at decoration / registration time and regardless of the configured backend: +interop keys are portable, so a namespace valid on one backend is valid on all. The +reservation is exact-match and namespace-only — `nsapix` is a valid namespace, and `ns` +and `nsapi` are valid operations. The `reject_reserved_namespace_*` error vectors and the +`reservation_scope` key vector pin it. + > [!WARNING] > **Full-string means full-string.** In Python, `re.match` with a `$` anchor still > accepts a trailing newline (`"users\n"` passes) — use `re.fullmatch`. A segment @@ -374,7 +386,8 @@ Two vectors substantiate this end-to-end, not just by construction: ## SaaS Considerations The SaaS API is format-agnostic — keys are opaque strings and values are opaque -bytes ([saas-api.md](saas-api.md)). Interop keys carry **no `ns:` prefix**; the +bytes ([saas-api.md](saas-api.md)). Interop keys carry **no `ns:` or `nsapi:` prefix** +— the reserved namespaces in [Segment grammar](#segment-grammar) guarantee it — so the `{namespace}` segment is an SDK-level convention, not a SaaS routing element (tenant isolation comes from authentication, not key parsing). @@ -385,8 +398,10 @@ isolation comes from authentication, not key parsing). > accepts interop-format keys; see > [cache-key-format.md → Server-Side Requirements](cache-key-format.md#server-side-requirements). > The interop segment grammar (lowercase, no `:` beyond the two delimiters, no `/`, -> max 194 chars) is deliberately a strict subset of what the security-only -> validator accepts. +> max 194 chars, no reserved namespace) is deliberately a subset of what the +> security-only validator accepts, with one known exception: the grammar admits `..` +> inside a segment, and the validator rejects `..` anywhere in a key (the Traversal +> row), so such a key fails with `400`. --- @@ -422,7 +437,8 @@ const getUser = cache.wrap(fetchUser, { An SDK implementation of interop mode MUST: -1. Require explicit `namespace` and `operation`, validated against the segment grammar. +1. Require explicit `namespace` and `operation`, validated against the segment grammar + (including the reserved namespaces `ns` and `nsapi`). 2. Build the canonical argument array per the binding rules (named→positional, defaults applied where introspectable). 3. Normalize and encode per this spec; reject out-of-model values with an error. @@ -471,11 +487,11 @@ not re-litigated by accident. | Group | Count | Verifies | | :--- | :---: | :--- | -| `key_vectors` | 33 | Canonical argument bytes (exact hex), args hash, full key — the `2.0`≡`2` collapse pair, supplementary-plane key sorting, heterogeneous and mixed-sign sets (byte order ≠ natural order), set dedupe (`{2, 2.0}` → `[2]`), datetime edge cases incl. pre-epoch, both collapse-range endpoints, and every `*16`-tier width boundary (uint/int ladders, str/bin/array/map headers, root array16) | +| `key_vectors` | 34 | Canonical argument bytes (exact hex), args hash, full key — the `2.0`≡`2` collapse pair, supplementary-plane key sorting, heterogeneous and mixed-sign sets (byte order ≠ natural order), set dedupe (`{2, 2.0}` → `[2]`), datetime edge cases incl. pre-epoch, both collapse-range endpoints, every `*16`-tier width boundary (uint/int ladders, str/bin/array/map headers, root array16), and the reservation's exact-match, namespace-only scope (`nsapix` namespace, `nsapi` operation) | | `value_vectors` | 4 | Plain-MessagePack value bytes (exact hex), float64 preservation in the value profile, temporal sentinel maps | | `aad_vectors` | 1 | AAD v0x03 bytes over an interop key (`format=msgpack`, `compressed=False`) | | `encryption_vectors` | 1 | Full HKDF-SHA256 → AES-256-GCM round-trip over plain-msgpack plaintext with the interop AAD (fixed nonce; decrypt-verified) | -| `error_vectors` | 9 | Inputs that MUST be rejected (NaN, +Inf and −Inf as independent vectors, int overflow/underflow, naive datetime, bad segments incl. trailing newline). The `error` text is a maintainer note, not a normative message | +| `error_vectors` | 11 | Inputs that MUST be rejected (NaN, +Inf and −Inf as independent vectors, int overflow/underflow, naive datetime, bad segments incl. trailing newline, the reserved namespaces `ns` and `nsapi`). The `error` text is a maintainer note, not a normative message | Inputs use a tagged-JSON convention (`{"$set": …}`, `{"$float": "2.0"}`, `{"$int": "…"}`, `{"$datetime": "…"}`, `{"$uuid": "…"}`, `{"$bytes": ""}`) diff --git a/test-vectors/interop-mode.json b/test-vectors/interop-mode.json index 10eaed3..9da1654 100644 --- a/test-vectors/interop-mode.json +++ b/test-vectors/interop-mode.json @@ -1,12 +1,12 @@ { - "version": "1.0.0", + "version": "1.1.0", "spec": "spec/interop-mode.md", "generator": "tools/interop-reference.py (CPython stdlib)", "cross_checked_by": "tools/interop-crosscheck.mjs (independent encoder + @noble/hashes blake2b + WebCrypto HKDF/AES-GCM)", "hash_algorithm": "blake2b-256 (digest_size=32, unkeyed) over canonical MessagePack of the flat argument array", "key_format": "{namespace}:{operation}:{args_hash}", "segment_pattern": "^[a-z0-9][a-z0-9._-]{0,63}$", - "segment_pattern_note": "Full-string match REQUIRED (Python: re.fullmatch, not re.match \u2014 $ matches before a trailing newline).", + "segment_pattern_note": "Full-string match REQUIRED (Python: re.fullmatch, not re.match \u2014 $ matches before a trailing newline). namespace additionally MUST NOT be exactly 'ns' or 'nsapi' (reserved: the server parses those key prefixes). The reservation is namespace-only; operation has no reserved values.", "width_coverage_note": "All *16 header boundaries (uint/int widths, str8->str16, bin8->bin16, fixarray->array16, fixmap->map16, including the root argument array) are pinned by vectors. The *32 tier (str32/bin32/array32/map32, >=64 KiB or >=65536 elements) is normative and implemented by both tools but untested-by-design: fixture blobs that size would bloat the file without exercising different logic (same length-prefix code path, wider field).", "error_vectors_note": "The 'error' field is a human-readable reason for maintainers. Conformance means the input MUST be rejected with an error; the message text is not normative.", "tagged_json": { @@ -593,6 +593,18 @@ "canonical_args_hex": "932aa568656c6c6f82a16101a16202", "args_hash": "03a0edbae0c1b5816c431652268d1527730175cdc0b45807cb07e1025ff971f6", "expected_key": "t:op:03a0edbae0c1b5816c431652268d1527730175cdc0b45807cb07e1025ff971f6" + }, + { + "name": "reservation_scope", + "description": "The ns/nsapi reservation is exact-match and namespace-only: namespace 'nsapix' (rejected by an ns* or nsapi* prefix match) and operation 'nsapi' stay valid", + "namespace": "nsapix", + "operation": "nsapi", + "args": [ + 1 + ], + "canonical_args_hex": "9101", + "args_hash": "405f09a3617bcc1425ea95b9840d9c2713e3ecebd5a3227abc599317b732e21a", + "expected_key": "nsapix:nsapi:405f09a3617bcc1425ea95b9840d9c2713e3ecebd5a3227abc599317b732e21a" } ], "value_vectors": [ @@ -712,6 +724,20 @@ "operation": "get_user", "args": [], "error": "segment validation must be a FULL-string match (Python re.match + $ accepts a trailing newline; use fullmatch)" + }, + { + "name": "reject_reserved_namespace_ns", + "namespace": "ns", + "operation": "get_user", + "args": [], + "error": "namespace 'ns' is reserved: the server parses a key starting 'ns:' as namespace-prefixed (here it would scope the key to a namespace named 'get_user')" + }, + { + "name": "reject_reserved_namespace_nsapi", + "namespace": "nsapi", + "operation": "users.fetch_by_id", + "args": [], + "error": "namespace 'nsapi' is reserved: the server parses a key starting 'nsapi:' as namespace-prefixed (rejected whatever the operation, including one the server would 400 on for its '.')" } ], "aad_vectors": [ diff --git a/tools/interop-crosscheck.mjs b/tools/interop-crosscheck.mjs index 64cf8d2..7731396 100644 --- a/tools/interop-crosscheck.mjs +++ b/tools/interop-crosscheck.mjs @@ -269,6 +269,14 @@ const here = dirname(fileURLToPath(import.meta.url)); const vectorsPath = process.argv[2] ?? join(here, "..", "test-vectors", "interop-mode.json"); const doc = JSON.parse(readFileSync(vectorsPath, "utf8")); +// Segment grammar: the fixture's pattern governs both segments; the reserved +// namespaces are hard-coded from the spec, not read from the fixture, so the +// reservation is checked by a second implementation rather than echoed back. +const segmentRe = new RegExp(doc.segment_pattern, "u"); +const RESERVED_NAMESPACES = new Set(["ns", "nsapi"]); +const segmentsValid = (namespace, operation) => + segmentRe.test(namespace) && segmentRe.test(operation) && !RESERVED_NAMESPACES.has(namespace); + let failures = 0; const check = (name, kind, expected, actual) => { if (expected !== actual) { @@ -278,6 +286,7 @@ const check = (name, kind, expected, actual) => { }; for (const v of doc.key_vectors) { + check(v.name, "segments valid", true, segmentsValid(v.namespace, v.operation)); const args = fromTagged(v.args); const bytes = encodeToBuffer(args, { collapseFloats: true }); check(v.name, "canonical_args_hex", v.canonical_args_hex, bytes.toString("hex")); @@ -357,9 +366,8 @@ for (const v of doc.encryption_vectors ?? []) { for (const v of doc.error_vectors) { try { - if (v.namespace !== undefined) { - const re = new RegExp(doc.segment_pattern, "u"); - if (!re.test(v.namespace) || !re.test(v.operation)) throw new Error("segment rejected"); + if (v.namespace !== undefined && !segmentsValid(v.namespace, v.operation)) { + throw new Error("segment rejected"); } encodeToBuffer(fromTagged(v.args), { collapseFloats: true }); failures++; diff --git a/tools/interop-reference.py b/tools/interop-reference.py index 2b5c2dc..529ac2e 100644 --- a/tools/interop-reference.py +++ b/tools/interop-reference.py @@ -40,6 +40,12 @@ # Implementations MUST full-string match (Python re.match would accept a # trailing newline because $ matches before it — use fullmatch, never match). SEGMENT_RE = re.compile(r"^[a-z0-9][a-z0-9._-]{0,63}$") +# Exact namespace values the grammar admits but the SaaS server parses as a key +# prefix (spec/cache-key-format.md#server-side-requirements): a key starting +# `ns:` or `nsapi:` would be scoped to a namespace named after the operation, or +# rejected. Namespace-only and exact-match — `ns` as an operation, or `nsapix` as +# a namespace, cannot form either prefix. +RESERVED_NAMESPACES = frozenset({"ns", "nsapi"}) UINT64_MAX = 2**64 - 1 INT64_MIN = -(2**63) @@ -245,6 +251,11 @@ def interop_key(namespace: str, operation: str, args: list | tuple) -> str: raise InteropError( f"invalid interop {name} {seg!r}: must full-string match ^[a-z0-9][a-z0-9._-]{{0,63}}$" ) + if namespace in RESERVED_NAMESPACES: + raise InteropError( + f"invalid interop namespace {namespace!r}: 'ns' and 'nsapi' are reserved " + "(the server parses a key starting 'ns:' or 'nsapi:' as namespace-prefixed)" + ) return f"{namespace}:{operation}:{args_hash(args)}" @@ -585,6 +596,16 @@ def tagged_args(raw: list) -> list: "operation": "op", "args": [42, "hello", {"b": 2, "a": 1}], }, + { + "name": "reservation_scope", + "description": ( + "The ns/nsapi reservation is exact-match and namespace-only: namespace 'nsapix' " + "(rejected by an ns* or nsapi* prefix match) and operation 'nsapi' stay valid" + ), + "namespace": "nsapix", + "operation": "nsapi", + "args": [1], + }, ] VALUE_VECTORS: list[dict] = [ @@ -658,6 +679,26 @@ def tagged_args(raw: list) -> list: "args": [], "error": "segment validation must be a FULL-string match (Python re.match + $ accepts a trailing newline; use fullmatch)", }, + { + "name": "reject_reserved_namespace_ns", + "namespace": "ns", + "operation": "get_user", + "args": [], + "error": ( + "namespace 'ns' is reserved: the server parses a key starting 'ns:' as namespace-prefixed " + "(here it would scope the key to a namespace named 'get_user')" + ), + }, + { + "name": "reject_reserved_namespace_nsapi", + "namespace": "nsapi", + "operation": "users.fetch_by_id", + "args": [], + "error": ( + "namespace 'nsapi' is reserved: the server parses a key starting 'nsapi:' as namespace-prefixed " + "(rejected whatever the operation, including one the server would 400 on for its '.')" + ), + }, ] @@ -676,7 +717,7 @@ def _build() -> dict: "args": v["args"], "canonical_args_hex": cab.hex(), "args_hash": h, - "expected_key": f"{v['namespace']}:{v['operation']}:{h}", + "expected_key": interop_key(v["namespace"], v["operation"], args), } ) @@ -700,14 +741,18 @@ def _build() -> dict: aad = aad_v3(ENC_TENANT_ID, single_int["expected_key"]) return { - "version": "1.0.0", + "version": "1.1.0", "spec": "spec/interop-mode.md", "generator": "tools/interop-reference.py (CPython stdlib)", "cross_checked_by": "tools/interop-crosscheck.mjs (independent encoder + @noble/hashes blake2b + WebCrypto HKDF/AES-GCM)", "hash_algorithm": "blake2b-256 (digest_size=32, unkeyed) over canonical MessagePack of the flat argument array", "key_format": "{namespace}:{operation}:{args_hash}", "segment_pattern": "^[a-z0-9][a-z0-9._-]{0,63}$", - "segment_pattern_note": "Full-string match REQUIRED (Python: re.fullmatch, not re.match — $ matches before a trailing newline).", + "segment_pattern_note": ( + "Full-string match REQUIRED (Python: re.fullmatch, not re.match — $ matches before a trailing newline). " + "namespace additionally MUST NOT be exactly 'ns' or 'nsapi' (reserved: the server parses those key " + "prefixes). The reservation is namespace-only; operation has no reserved values." + ), "width_coverage_note": ( "All *16 header boundaries (uint/int widths, str8->str16, bin8->bin16, fixarray->array16, " "fixmap->map16, including the root argument array) are pinned by vectors. The *32 tier "