@@ -41,8 +41,7 @@ This document specifies two layers:
4141 decode byte-identity for every vector and re-encode byte-identity for the
4242 canonical ` *_bin ` vectors only — legacy array-of-integers vectors are
4343 decode-only, retained as legacy-read proof. That re-encode assertion covers
44- only the vectors the pinned file contains (core currently vendors 1.1.0, with
45- the resulting gap detailed below). Byte-canonicity scopes to the
44+ only the vectors the pinned file contains. Byte-canonicity scopes to the
4645 envelope's MessagePack encoding and to the ** canonical writer's** output:
4746 the LZ4 bytes inside ` compressed_data ` are not reproducible across
4847 conforming compressors — see
@@ -287,31 +286,23 @@ bytes are therefore
287286 valid LZ4 block satisfies, so it accepts a re-pin to unrelated bytes. Neither
288287 half runs ` lz4_flex ` , so neither can detect an ` lz4_flex ` ** behaviour** change;
289288 that remains the job of the re-encode assertions in ` cachekit-core ` described
290- below, subject to the vendored-version gap noted there .
289+ below.
291290
292291This is the same doctrine [ interop v2] ( interop-v2.md ) records for its
293292compressed-values profile. The pinned bytes are the ** canonical implementation's**
294293output (` lz4_flex ` via ` cachekit-core ` ), enforced by the re-encode byte-identity
295- assertions in ` cachekit-core/tests/wire_format_vectors.rs ` — ** but only for the
296- vectors present in the fixture that repo vendors** . That matters today:
297- cachekit-core vendors 1.1.0 and pins ` version == "1.1.0" ` , so
298- ` width_boundary_bin16 ` (added at 1.1.1) has ** no canonical-writer (` lz4_flex ` )
299- compressed-byte check anywhere in the fleet** , and its pinned xxh3-64 checksum
300- is recomputed nowhere. Its MessagePack encoding * is* covered: this repo's
301- ` tools/wire-format-reference.py verify ` asserts legacy and bin re-encode
302- byte-identity for it on every run, and liblz4 reproduces its compressed bytes
303- on the optional ` lz4 ` leg — so do not read this gap as "the vector is
304- unverified". Closing it means re-vendoring 1.1.1 into cachekit-core, which
305- requires three changes together, not one: bump ` FIXTURE_SHA256 ` , bump the
306- ` version == "1.1.0" ` pin to ` 1.1.1 ` , and relax
307- ` assert_eq!(twin_bytes[1], 0xc4) ` to accept ` 0xc5 ` — that assertion currently
308- requires * every* twin to be bin8, and ` width_boundary_bin16_bin ` is bin16
309- (marker ` 0xc5 ` , 303-byte ` compressed_data ` ), which is the whole point of the
310- vector. A drop-in re-vendor fails that test. The reference liblz4 mapping
311- above (` lz4.block ` ) is ** decode-verified against every vector** in this repo's
312- CI (` tools/wire-format-reference.py verify ` , optional ` lz4 ` leg); on encode it
313- reproduces every pair except ` large_compressible ` byte-for-byte, which is an
314- observation, not a guarantee — but one this repo's CI pins (see
294+ assertions in ` cachekit-core/tests/wire_format_vectors.rs ` — ** for the vectors
295+ present in the fixture that repo vendors** , which recompute each twin's
296+ ` lz4_flex ` bytes and xxh3-64 checksum. Anyone vendoring the fixture should
297+ derive each ` *_bin ` twin's expected marker from its decoded ` compressed_data `
298+ length (` ≤255 → 0xc4 ` , ` ≤65535 → 0xc5 ` , else ` 0xc6 ` ), as cachekit-core does. An
299+ assertion that every twin is bin8 fails on ` width_boundary_bin16_bin ` (` 0xc5 ` ,
300+ 303-byte ` compressed_data ` ), and one that accepts all three widths cannot
301+ detect a non-shortest header. The reference liblz4 mapping above (` lz4.block ` ) is ** decode-verified against
302+ every vector** in this repo's CI (` tools/wire-format-reference.py verify ` ,
303+ optional ` lz4 ` leg); on encode it reproduces every pair except
304+ ` large_compressible ` byte-for-byte, which is an observation, not a guarantee —
305+ but one this repo's CI pins (see
315306` LZ4_ENCODE_DIVERGENT ` ), so a toolchain change that alters the divergent set
316307fails CI rather than quietly making this paragraph wrong.
317308
0 commit comments