Decode container entries in DAG-JSON output - #20
Conversation
`view --json` on a container printed its entries as opaque bytes, taken from a re-encode of the decoded container. That re-encode sorts entries bytewise and drops the ones that decode as no known token kind, so both the entry count and the index `-i` takes could disagree with the input. The entries are now read out of the input and decoded in place, so `--json` reports what the file holds: - `jq '.["ctn-v1"] | length'` counts the tokens in one command - an array index selects the same token as `--container-index` - an entry that decodes as no known token kind is written as its bytes rather than dropped Stripping the transport encoding duplicates the codec handling that `container.Decode` keeps unexported. fil-forge/ucantone#55 exports it, and the copy goes away once that lands and the dependency is bumped. Assisted-by: Claude:claude-opus-5[1m] Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>
| ##### JSON output | ||
|
|
||
| The `--json` flag will output `dag-json` encoding of the input. | ||
| The `--json` flag outputs the `dag-json` encoding of the input. For a container, the entries are decoded in place and appear in the order they have in the input, so an array index selects the same token as `--container-index`. |
There was a problem hiding this comment.
I asked Claude to explain this breaking change:
The | jq in the README is cosmetic — the tool prints one line either way, jq just pretty-prints it. Not a change.
The entry shape is a breaking change. .["ctn-v1"][N] was {"/":{"bytes":"…"}} and is now [sig, {tag: payload}]. Verified consequence:
old output → container.UnmarshalDagJSON:
new output → container.UnmarshalDagJSON: expected object open but read [
There's a second problem underneath it. --json is documented as "the dag-json encoding of the input", and for a container the ctn-v1 field genuinely is a list of byte strings. So the new output is valid DAG-JSON but it is not the DAG-JSON encoding of the container. My change quietly redefines what the flag means, and the mixed-shape array (token envelope or raw bytes) is a wart that only exists because of that redefinition.
Worth noting the entry-source fix is separable and not breaking on its own: reading entries from the input instead of the roundtrip fixes the order, the count and the -i disagreement while keeping the byte-string shape.
Written by Claude.
view --jsonon a container prints its entries as opaque bytes, so reading anything out of a container takes oneview -i Nper entry and there is no way to learn the entry count except walking-iuntil it errors.The entries are now decoded in place:
One behaviour change worth review
The entries come out of the input now. They used to come from a re-encode of the decoded container, and that roundtrip sorts entries bytewise and silently drops the ones
decodeTokenscannot read, with a nil error. A container holding three entries where one is junk printed two and said nothing, and-jentry N and-i Ncould be different tokens.So three things move for a container whose entries were not already sorted, or that holds an entry this tool cannot decode: the
--jsonarray, theviewtable, and the root CID the table prints. Containers this tool produces are already sorted, so nothing moves on the happy path.An entry that decodes as no known token kind is written as its bytes,
{"/":{"bytes":"…"}}. That keeps the entry count and every index aligned with the input, and it makes the array mixed-shape: a caller walking it needs a type check to tell a token from an undecodable entry. The alternative,null, loses the bytes; dropping it is what this change is fixing.The output is a view rather than a re-encoding. It no longer round-trips back through
UnmarshalDagJSON, which took the entries as byte strings.Duplication to remove later
decodeContainerCBORstrips the codec byte, base64 and gzip, which duplicates about thirty lines thatcontainer.Decodekeeps unexported. fil-forge/ucantone#55 exports that step ascontainer.DecodeTransport; the copy goes away once it lands and the dependency is bumped. The bump crosses about eight weeks ofucantonehistory, so it is separate work.Relation to #19
#19 makes the same change to where the entries come from, because summarising every token needs the same raw-entry walk. Whichever lands first carries it, and the other drops that part on rebase. The features are independent: this one changes what
--jsonprints, #19 adds--summary.Tests
cmd/view_test.godrivesrootCmdthe waycmd/delegate_test.godoes, building fixtures withucantool delegaterather than committing any. It pins that every entry of a three-command container is present, that the entries are decoded rather than emitted as bytes, that a container with an undecodable entry reports both entries with the undecodable one as bytes at index 0, that an array index agrees with-ion every entry, and that the audience is reachable without naming the spec version in the path.