fix(sdk): DSPX-4590 zip64 conformance by dmihalcik-virtru · Pull Request #3981 · opentdf/platform

dmihalcik-virtru · 2026-09-04T16:29:29Z

Jira: https://virtru.atlassian.net/browse/DSPX-4590

Part 3 of a 3-PR stack: #3933 (base) <- #3979 <- this PR.

Based on #3979 (default per-segment sizes when a writer omits them), which
is itself based on #3933. This PR carries findings 1-6 of the DSPX-4590
investigation (go-sdk's ZIP64/APPNOTE conformance issues); finding 7
(per-segment size defaults) is fixed in #3979 below it and is unrelated to
these changes -- the two were originally one PR (#3967) and are split here
so each can be reviewed independently.

go-sdk's zip layer disagrees with the other SDKs in a handful of places,
and this addresses all six.

Finding 1 (interop): writer switched to ZIP64 at 4 GiB instead of 2 GiB

Finalize compared against ^uint32(0), so a payload between 2 GiB and 4 GiB
was written as a zip32 archive with a value in the top half of the unsigned
32-bit range. java-sdk widens those central-directory fields signed, so
deployed Java clients read the size/offset back as a negative number and cannot
open the container. web-sdk always writes ZIP64, java-sdk (since java-sdk#393)
switches at Integer.MAX_VALUE.

New maxNonZip64Value = math.MaxInt32 in zip_primitives.go, mirroring
java-sdk's MAX_NON_ZIP64_VALUE.
The rule is applied to the uncompressed size, the compressed size and
entry.Offset (the local-header offset), which was previously not checked
at all -- an archive under 2 GiB of payload could still place a later
entry's header past the boundary.
The threshold is injectable: Config.MaxNonZip64Value plus a
WithMaxNonZip64Value option (clamped to (0, maxNonZip64Value]), so the
tests can drive the ZIP64 path with a 1 KiB threshold instead of
allocating gigabytes.

Finding 2: ZIP64 extra field parsed positionally

The reader assumed the ZIP64 extended-information field was first in the
extra-field area and that all three values were always present. Per APPNOTE
4.5.3 the values appear in the order original size, compressed size, local
header offset, and each is present only when its central-directory
counterpart holds the 0xFFFFFFFF sentinel. parseZip64ExtraField now walks
the whole extra area, skips foreign tags, reads values in spec order gated on
the sentinels, and rejects a field that claims to run past the end of the
area.

Finding 3: ZIP64 detected from the CD offset alone

NewReader only looked at CentralDirectoryOffset == 0xFFFFFFFF. An archive
that overflows the entry count (0xFFFF) or the central-directory size but
not the offset was read as zip32. eocdNeedsZip64 now checks all three EOCD
sentinel fields.

Finding 4: per-entry ZIP64 extra field ignored without a ZIP64 EOCD

A writer may put a ZIP64 extra field on an individual entry while leaving
the EOCD in zip32 form. The reader now consults the extra field whenever the
central-directory header carries a sentinel, independent of the EOCD form.

Finding 5: central-directory cursor could wrap at uint16

nextCD was advanced with uint16 arithmetic and did not include the file
comment. A 65000-byte filename plus a 600-byte extra field wraps to 110 and
the reader walks into the middle of a header. The advance is now done in
uint64 and includes FileCommentLength.

Finding 6: silent truncation when a value does not fit 32 bits

The zip32 paths narrowed with a bare cast. checkFitsInCentralDirectory now
returns a new ErrFieldOverflow for the compressed size, uncompressed size,
local-header offset, central-directory size/offset and entry count instead
of writing a corrupt archive. (With finding 1 in place this is unreachable
in normal operation; it is a backstop against future callers.)

Tests

sdk/internal/zipstream/zip64_conformance_test.go -- hand-assembles raw zip
archives (buildRawZip) so the reader can be pointed at containers no Go
writer would produce: extra field not first, differing compressed and
uncompressed sizes so the APPNOTE 4.5.3 ordering is actually asserted, per-
entry ZIP64 under a zip32 EOCD, a central-directory file comment, the
65646-byte name+extra case that wraps to 110 at uint16, ZIP64 implied by the
entry count, and a malformed extra field. Writer side:
TestWriterSwitchesToZip64AtInjectedThreshold uses the injected 1 KiB
threshold, TestEntryNeedsZip64AtTwoGiB pins maxNonZip64Value == math.MaxInt32 and covers size / compressed size / offset,
TestCentralDirectoryNarrowingGuard covers finding 6.
sdk/internal/zipstream/fuzz_test.go -- two new seeds for the nextCD
overflow and the file-comment case.

Follow-up required in opentdf/tests (NOT covered by this PR)

The xtest cell test_tdfs.py::test_chunky_roundtrip currently SKIPS
for go, because xtest/sdk/go/cli.sh answers no to supports chunky.
That shim lives in the opentdf/tests repo, so merging this stack does
not flip it -- the go column will stay skipped and the interop
regression will stay invisible in CI. When this fix ships in a release,
someone needs to version-gate the chunky) case in xtest/sdk/go/cli.sh
so it reports support at or above that version.

Testing

cd sdk && go test ./... -race
make fmt, make lint (0 new issues)

Supersedes the zip64 portion of #3967, which is left open, unmodified, for
reference.

Summary by CodeRabbit

Bug Fixes
- Improved ZIP and ZIP64 compatibility for large archives, including archives with central-directory comments and varied ZIP64 metadata.
- Prevented incorrect size calculations and silent field truncation when archive values exceed standard ZIP limits.
- Added validation for malformed ZIP extra fields and unsupported field overflows.
New Features
- Added a configurable threshold for switching from standard ZIP format to ZIP64.
- Added an explicit error when archive values cannot fit in standard ZIP fields.

coderabbitai · 2026-09-04T16:29:37Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 7916ac41-ae59-46e1-9a0b-5effe5826bfd

📥 Commits

Reviewing files that changed from the base of the PR and between 9deeee8 and fa3a416.

📒 Files selected for processing (7)

sdk/internal/zipstream/fuzz_test.go
sdk/internal/zipstream/reader.go
sdk/internal/zipstream/segment_writer.go
sdk/internal/zipstream/writer.go
sdk/internal/zipstream/zip64_conformance_test.go
sdk/internal/zipstream/zip_headers.go
sdk/internal/zipstream/zip_primitives.go

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

Walkthrough

The ZIP stream implementation now improves ZIP64 detection and extra-field parsing, widens central-directory offset arithmetic, adds configurable ZIP64 thresholds, rejects field overflow, and adds conformance and fuzz coverage.

Changes

ZIP64 support

Layer / File(s)	Summary
Reader ZIP64 parsing `sdk/internal/zipstream/reader.go`, `sdk/internal/zipstream/zip64_conformance_test.go`, `sdk/internal/zipstream/fuzz_test.go`	The reader detects ZIP64 from central-directory size, offset, and entry-count sentinels. It parses ZIP64 extra fields in any extra-field position, validates malformed fields, and includes comments and wide lengths in central-directory advancement.
Writer ZIP64 thresholds and field checks `sdk/internal/zipstream/writer.go`, `sdk/internal/zipstream/zip_primitives.go`, `sdk/internal/zipstream/zip_headers.go`	The writer adds `MaxNonZip64Value`, `WithMaxNonZip64Value`, and `ErrFieldOverflow`. Central-directory entries and EOCD fields now use threshold checks and reject values that cannot fit.
Segment writer integration and conformance `sdk/internal/zipstream/segment_writer.go`, `sdk/internal/zipstream/zip64_conformance_test.go`	Segment finalization applies the configured ZIP64 threshold. Tests cover ZIP64 output, boundary values, entry counts, offsets, and overflow handling.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SegmentWriter
  participant CentralDirectory
  participant ZIP64Records
  SegmentWriter->>CentralDirectory: finalize entries with configured threshold
  CentralDirectory->>CentralDirectory: check sizes and offsets
  CentralDirectory-->>SegmentWriter: ZIP32 or ZIP64 decision
  SegmentWriter->>ZIP64Records: write ZIP64 records when required

Suggested reviewers: elizabethhealy

Merge Risk: ⚪ Minimal · up to 38472

The ZIP64 conformance fixes appear internally consistent and covered by focused tests; no actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies an SDK ZIP64 conformance fix and matches the main changes in the pull request.
Docstring Coverage	✅ Passed	Docstring coverage is 96.43% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 28 functions across 7 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch DSPX-4590-zip64-conformance-only

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{A rabbit reads each line,
The patch grows clear beneath the moon,
Small changes hop in place,
Tests guard the garden path,
Reviews bloom before the dawn.}

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-09-04T16:35:00Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	153.769214ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	78.340304ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	269.885813ms
Throughput	370.53 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	35.476608296s
Average Latency	354.023502ms
Throughput	140.94 requests/second

github-actions · 2026-09-04T16:38:07Z

X-Test Failure Report

opentdf~~platform~~UW0E3Q.dockerbuild
govulncheck-failure-8
govulncheck-failure-3
govulncheck-failure-1

github-actions · 2026-09-04T16:40:46Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	147.005046ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	87.962554ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	283.454448ms
Throughput	352.79 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	39.771002646s
Average Latency	396.829653ms
Throughput	125.72 requests/second

github-actions · 2026-09-04T16:41:26Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	221.215411ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	120.793391ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	437.020703ms
Throughput	228.82 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	56.487553578s
Average Latency	563.647924ms
Throughput	88.52 requests/second

github-actions · 2026-09-04T16:53:44Z

X-Test Results

✅ go@pull-3981-main
✅ go@pull-3981-pull-3981
✅ go@main-pull-3981
✅ go@main-main
✅ go@pull-3981-v0.9.0
bats-test-results
✅ go@main-v0.9.0
✅ java@main-main
✅ js@main-pull-3981
✅ java@main-v0.9.0
✅ java@main-pull-3981
cukes-report
✅ js@main-v0.9.0
✅ js@main-main
opentdf~~platform~~K8ECVP.dockerbuild
govulncheck-failure-8
govulncheck-failure-1
govulncheck-failure-3

github-actions · 2026-09-04T20:25:24Z

X-Test Failure Report

github-actions · 2026-09-04T20:31:06Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	226.380952ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	124.051175ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	457.231154ms
Throughput	218.71 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	59.360582491s
Average Latency	592.476556ms
Throughput	84.23 requests/second

github-actions · 2026-09-04T20:31:08Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	195.744576ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	103.907686ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	367.41732ms
Throughput	272.17 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	51.529894041s
Average Latency	514.155542ms
Throughput	97.03 requests/second

github-actions · 2026-09-04T20:36:14Z

X-Test Failure Report

bats-test-results
govulncheck-failure-8
opentdf~~platform~~QD354O.dockerbuild
govulncheck-failure-1
govulncheck-failure-3

github-actions · 2026-09-04T20:39:39Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	219.251682ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	121.475667ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	446.480946ms
Throughput	223.97 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	55.794171102s
Average Latency	556.648353ms
Throughput	89.62 requests/second

github-actions · 2026-09-04T20:40:01Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	253.937424ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	138.927697ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	442.519697ms
Throughput	225.98 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	1m1.935929654s
Average Latency	617.897325ms
Throughput	80.73 requests/second

github-actions · 2026-09-04T20:53:34Z

X-Test Results

✅ go@pull-3981-main
✅ go@main-pull-3981
✅ go@pull-3981-pull-3981
✅ go@main-main
✅ go@pull-3981-v0.9.0
✅ go@main-v0.9.0
✅ java@main-pull-3981
bats-test-results
✅ java@main-v0.9.0
✅ js@main-pull-3981
✅ js@main-main
✅ java@main-main
✅ js@main-v0.9.0
cukes-report
opentdf~~platform~~M6Y0FD.dockerbuild
govulncheck-failure-8
govulncheck-failure-3
govulncheck-failure-1

github-actions · 2026-09-08T14:19:51Z

X-Test Failure Report

github-actions · 2026-09-08T14:25:52Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	236.767957ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	133.537011ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	421.29572ms
Throughput	237.36 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	58.749137658s
Average Latency	585.783907ms
Throughput	85.11 requests/second

github-actions · 2026-09-08T14:25:56Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	237.007169ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	147.413645ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	422.50271ms
Throughput	236.68 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	1m1.859583049s
Average Latency	617.177698ms
Throughput	80.83 requests/second

github-actions · 2026-09-08T14:33:23Z

X-Test Results

bats-test-results
✅ java@main-main
✅ go@main-v0.9.0
✅ go@main-pull-3981
✅ go@main-main
✅ go@pull-3981-v0.9.0
✅ go@pull-3981-main
✅ java@main-pull-3981
✅ go@pull-3981-pull-3981
cukes-report
✅ js@main-main
✅ java@main-v0.9.0
✅ js@main-pull-3981
✅ js@main-v0.9.0
opentdf~~platform~~1USRS9.dockerbuild
govulncheck-failure-8
govulncheck-failure-3
govulncheck-failure-1

github-actions · 2026-09-08T17:01:43Z

X-Test Results

bats-test-results
✅ go@main-main
✅ go@pull-3981-main
✅ go@main-pull-3981
✅ go@pull-3981-pull-3981
✅ go@main-v0.9.0
✅ go@pull-3981-v0.9.0
✅ java@main-pull-3981
cukes-report
✅ js@main-pull-3981
✅ js@main-v0.9.0
✅ java@main-main
✅ js@main-main
✅ java@main-v0.9.0
opentdf~~platform~~3SK4CQ.dockerbuild
govulncheck-failure-8
govulncheck-failure-3
govulncheck-failure-1

github-actions · 2026-09-08T19:02:27Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	247.717128ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	146.585892ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	432.788063ms
Throughput	231.06 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	1m0.064043048s
Average Latency	599.294891ms
Throughput	83.24 requests/second

github-actions · 2026-09-08T19:09:33Z

X-Test Results

✅ go@main-main
✅ go@main-pull-3981
✅ go@pull-3981-main
✅ go@pull-3981-v0.9.0
✅ go@main-v0.9.0
✅ go@pull-3981-pull-3981
cukes-report
✅ java@main-pull-3981
✅ java@main-main
✅ js@main-main
✅ js@main-pull-3981
✅ js@main-v0.9.0
✅ java@main-v0.9.0
opentdf~~platform~~V5AWD5.dockerbuild
govulncheck-failure-1
govulncheck-failure-3
govulncheck-failure-8

github-actions · 2026-09-11T04:31:38Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	222.853451ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	124.481119ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	446.674739ms
Throughput	223.88 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	57.764648046s
Average Latency	576.392165ms
Throughput	86.56 requests/second

github-actions · 2026-09-11T04:40:46Z

X-Test Results

✅ go@main-main
✅ go@pull-3981-main
✅ go@pull-3981-pull-3981
✅ go@main-pull-3981
✅ go@pull-3981-v0.9.0
cukes-report
✅ go@main-v0.9.0
✅ java@main-pull-3981
✅ java@main-main
bats-test-results
✅ java@main-v0.9.0
✅ js@main-main
✅ js@main-v0.9.0
✅ js@main-pull-3981
opentdf~~platform~~NYNH5Q.dockerbuild
govulncheck-failure-8
govulncheck-failure-3
govulncheck-failure-1

github-actions · 2026-09-14T17:02:45Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	262.371353ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	137.621995ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	423.553276ms
Throughput	236.10 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	1m0.260813075s
Average Latency	601.078808ms
Throughput	82.97 requests/second

github-actions · 2026-09-14T17:17:48Z

X-Test Results

✅ go@pull-3981-pull-3981
cukes-report
✅ js@main-v0.9.0
✅ go@main-pull-3981
✅ go@main-v0.9.0
✅ js@main-pull-3981
✅ java@main-main
✅ java@main-pull-3981
✅ java@main-v0.9.0
govulncheck-failure-1
✅ go@pull-3981-v0.9.0
✅ go@main-main
✅ go@pull-3981-main
opentdf~~platform~~UM7H6F.dockerbuild
govulncheck-failure-3
✅ js@main-main
bats-test-results
govulncheck-failure-8

go-sdk's zip layer disagrees with the other SDKs in a handful of places. This addresses findings 1-6 from the DSPX-4590 investigation (finding 7, per-segment size defaults, is the parent PR this one is stacked on). ## Finding 1 (interop): writer switched to ZIP64 at 4 GiB instead of 2 GiB `Finalize` compared against `^uint32(0)`, so a payload between 2 GiB and 4 GiB was written as a zip32 archive with a value in the top half of the unsigned 32-bit range. java-sdk widens those central-directory fields *signed*, so deployed Java clients read the size/offset back as a negative number and cannot open the container. web-sdk always writes ZIP64, java-sdk (since java-sdk#393) switches at `Integer.MAX_VALUE`. - New `maxNonZip64Value = math.MaxInt32` in `zip_primitives.go`, mirroring java-sdk's `MAX_NON_ZIP64_VALUE`. - The rule is applied to the uncompressed size, the compressed size **and** `entry.Offset` (the local-header offset), which was previously not checked at all -- an archive under 2 GiB of payload could still place a later entry's header past the boundary. - The threshold is injectable: `Config.MaxNonZip64Value` plus a `WithMaxNonZip64Value` option (clamped to `(0, maxNonZip64Value]`, so it can only ever make the writer *more* eager to use ZIP64). This lets the tests drive the ZIP64 path with a 1 KiB threshold instead of allocating gigabytes. ## Finding 2: ZIP64 extra field parsed positionally The reader assumed the ZIP64 extended-information field was the first entry in the extra-field area and that all three values were always present. Per APPNOTE 4.5.3 the values appear in the order *original size, compressed size, local header offset*, and each is present **only** when its central-directory counterpart holds the `0xFFFFFFFF` sentinel. A container whose extra area leads with, say, an extended-timestamp field (tag `0x5455`) was misparsed. `parseZip64ExtraField` now walks the whole extra area, skips foreign tags, reads values in spec order gated on the sentinels, and rejects a field that claims to run past the end of the area. ## Finding 3: ZIP64 detected from the CD offset alone `NewReader` only looked at `CentralDirectoryOffset == 0xFFFFFFFF`. An archive that overflows the entry count (`0xFFFF`) or the central-directory size but not the offset was read as zip32. `eocdNeedsZip64` now checks all three EOCD sentinel fields. ## Finding 4: per-entry ZIP64 extra field ignored without a ZIP64 EOCD A writer may put a ZIP64 extra field on an individual entry while leaving the EOCD in zip32 form. The reader now consults the extra field whenever the central-directory header carries a sentinel, independent of the EOCD form. ## Finding 5: central-directory cursor could wrap at uint16 `nextCD` was advanced with uint16 arithmetic and did not include the file comment. A 65000-byte filename plus a 600-byte extra field wraps to 110 and the reader walks into the middle of a header. The advance is now done in uint64 and includes `FileCommentLength`. ## Finding 6: silent truncation when a value does not fit 32 bits The zip32 paths narrowed with a bare cast. `checkFitsInCentralDirectory` now returns a new `ErrFieldOverflow` for the compressed size, uncompressed size, local-header offset, central-directory size/offset and entry count instead of writing a corrupt archive. (With finding 1 in place this is unreachable in normal operation; it is a backstop against future callers.) ## Tests `sdk/internal/zipstream/zip64_conformance_test.go` -- hand-assembles raw zip archives (`buildRawZip`) so the reader can be pointed at containers no Go writer would produce: extra field not first, differing compressed and uncompressed sizes so the APPNOTE 4.5.3 ordering is actually asserted (a fixture with equal sizes passes either way), per-entry ZIP64 under a zip32 EOCD, a central-directory file comment, the 65646-byte name+extra case that wraps to 110 at uint16, ZIP64 implied by the entry count, and a malformed extra field. Writer side: `TestWriterSwitchesToZip64AtInjectedThreshold` uses the injected 1 KiB threshold, `TestEntryNeedsZip64AtTwoGiB` pins `maxNonZip64Value == math.MaxInt32` and covers size / compressed size / offset, `TestCentralDirectoryNarrowingGuard` covers finding 6. `sdk/internal/zipstream/fuzz_test.go` -- two new seeds for the `nextCD` overflow and the file-comment case. ## Follow-up required in opentdf/tests (NOT covered by this PR) > The xtest cell `test_tdfs.py::test_chunky_roundtrip` currently **SKIPS** for > go, because `xtest/sdk/go/cli.sh` answers no to `supports chunky`. That shim > lives in the `opentdf/tests` repo, so merging this PR does **not** flip it -- > the go column will stay skipped and the interop regression will stay > invisible in CI. When this fix ships in a release, someone needs to > version-gate the `chunky)` case in `xtest/sdk/go/cli.sh` so it reports > support at or above that version. Signed-off-by: Dave Mihalcik <dmihalcik@virtru.com>

Invalidated by push of 38472ff

github-actions · 2026-09-14T18:15:45Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	242.442927ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	136.741059ms

Benchmark Statistics

Name	№ Requests	Avg Duration	Min Duration	Max Duration

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	427.795137ms
Throughput	233.76 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	58.834843079s
Average Latency	586.917956ms
Throughput	84.98 requests/second

github-actions · 2026-09-14T18:16:30Z

⚠️ Govulncheck found vulnerabilities ⚠️

The following modules have known vulnerabilities:

otdfctl
service
tests-bdd

See the workflow run for details.

github-actions · 2026-09-14T18:28:04Z

X-Test Results

bats-test-results
✅ java@main-pull-3981
✅ go@pull-3981-pull-3981
✅ java@main-main
✅ js@main-main
✅ java@main-v0.9.0
govulncheck-failure-1
✅ js@main-pull-3981
govulncheck-failure-3
✅ js@main-v0.9.0
govulncheck-failure-8
✅ go@main-v0.9.0
✅ go@pull-3981-main
cukes-report
✅ go@main-main
✅ go@pull-3981-v0.9.0
✅ go@main-pull-3981
opentdf~~platform~~GOCXTJ.dockerbuild

dmihalcik-virtru mentioned this pull request Sep 4, 2026

fix(sdk): DSPX-4590 default per-segment sizes when a writer omits them #3979

Merged

github-actions Bot added comp:sdk A software development kit, including library, for client applications and inter-service communicati size/m labels Sep 4, 2026

dmihalcik-virtru force-pushed the DSPX-4590-zip64-conformance-only branch from f57c73b to 01d6d43 Compare September 4, 2026 16:36

dmihalcik-virtru force-pushed the DSPX-4590-zip64-conformance-only branch from 01d6d43 to eaf68f8 Compare September 4, 2026 20:25

dmihalcik-virtru force-pushed the DSPX-4590-zip64-conformance-only branch from eaf68f8 to 2e0d5ec Compare September 4, 2026 20:34

dmihalcik-virtru force-pushed the DSPX-4590-zip64-conformance-only branch from 2e0d5ec to 67a7320 Compare September 8, 2026 14:19

Base automatically changed from DSPX-4590-zip64-conformance-v2 to main September 8, 2026 18:55

dmihalcik-virtru force-pushed the DSPX-4590-zip64-conformance-only branch from 80e1538 to fa3a416 Compare September 8, 2026 18:55

dmihalcik-virtru marked this pull request as ready for review September 10, 2026 14:17

dmihalcik-virtru requested review from a team as code owners September 10, 2026 14:17

dmihalcik-virtru force-pushed the DSPX-4590-zip64-conformance-only branch from fa3a416 to 7a4ca54 Compare September 11, 2026 04:25

dmihalcik-virtru requested a review from sujankota September 11, 2026 17:33

dmihalcik-virtru force-pushed the DSPX-4590-zip64-conformance-only branch from 7a4ca54 to dc98348 Compare September 14, 2026 16:56

sujankota previously approved these changes Sep 14, 2026

View reviewed changes

dmihalcik-virtru mentioned this pull request Sep 14, 2026

fix(sdk): bound zip64 reader offsets against the archive (DSPX-4590) #4043

Open

dmihalcik-virtru force-pushed the DSPX-4590-zip64-conformance-only branch from dc98348 to 38472ff Compare September 14, 2026 18:09

dmihalcik-virtru requested a review from sujankota September 14, 2026 18:14

sujankota approved these changes Sep 14, 2026

View reviewed changes

dmihalcik-virtru added this pull request to the merge queue Sep 14, 2026

Merged via the queue into main with commit 85bbe96 Sep 14, 2026
47 checks passed

dmihalcik-virtru deleted the DSPX-4590-zip64-conformance-only branch September 14, 2026 21:17

opentdf-automation Bot mentioned this pull request Sep 14, 2026

chore(main): release sdk 0.32.0 #3963

Merged

Uh oh!

Conversation

dmihalcik-virtru commented Sep 4, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Finding 1 (interop): writer switched to ZIP64 at 4 GiB instead of 2 GiB

Finding 2: ZIP64 extra field parsed positionally

Finding 3: ZIP64 detected from the CD offset alone

Finding 4: per-entry ZIP64 extra field ignored without a ZIP64 EOCD

Finding 5: central-directory cursor could wrap at uint16

Finding 6: silent truncation when a value does not fit 32 bits

Tests

Follow-up required in opentdf/tests (NOT covered by this PR)

Testing

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

github-actions Bot commented Sep 4, 2026

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Benchmark Statistics

Bulk Benchmark Results

TDF3 Benchmark Results:

Uh oh!

github-actions Bot commented Sep 4, 2026

X-Test Failure Report

Uh oh!

github-actions Bot commented Sep 4, 2026

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Benchmark Statistics

Bulk Benchmark Results

TDF3 Benchmark Results:

Uh oh!

github-actions Bot commented Sep 4, 2026

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Benchmark Statistics

Bulk Benchmark Results

TDF3 Benchmark Results:

Uh oh!

github-actions Bot commented Sep 4, 2026

X-Test Results

Uh oh!

github-actions Bot commented Sep 4, 2026

X-Test Failure Report

Uh oh!

github-actions Bot commented Sep 4, 2026

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Benchmark Statistics

Bulk Benchmark Results

TDF3 Benchmark Results:

Uh oh!

github-actions Bot commented Sep 4, 2026

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Benchmark Statistics

Bulk Benchmark Results

TDF3 Benchmark Results:

Uh oh!

github-actions Bot commented Sep 4, 2026

X-Test Failure Report

Uh oh!

github-actions Bot commented Sep 4, 2026

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Benchmark Statistics

Bulk Benchmark Results

TDF3 Benchmark Results:

Uh oh!

github-actions Bot commented Sep 4, 2026

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Benchmark Statistics

Bulk Benchmark Results

dmihalcik-virtru commented Sep 4, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 4, 2026 •

edited

Loading