fix(reader): harden 7 encodings against malformed adversarial input - #327
Merged
Conversation
dfa1
force-pushed
the
security/per-encoding-adversarial-tests
branch
from
August 5, 2026 19:50
6aef0de to
78e32b7
Compare
…gainst malformed input Part of TODO.md's "Per-encoding adversarial tests" security item (CLAUDE.md §Security contract): a malformed file must always throw VortexException, never a raw JDK exception. - DecodeContext.buffer(i)/decodeChild(i): centralize bounds checks on both the position and the segment/child index it holds. - VarBinArray (OffsetMode/DictMode/ViewMode): checkedLength() guards NegativeArraySizeException/IndexOutOfBoundsException from corrupted offsets; forEachByteLength paths get boundary catch-and-wrap without per-element branches. - DictEncodingDecoder: expand() guards the values-pool index and empty-child division by zero; legacy-dict path routed through the hardened buffer accessor instead of raw array indexing. - BitpackedEncodingDecoder: bit_width/offset range-checked upfront; packed-buffer-too-short wrapped at the unpack call site. - AlpEncodingDecoder: exponent table indices and patch indices range-checked; empty encoded/patch children guarded. - SparseEncodingDecoder: patch count bounded by row count; empty patch children rejected; patch-value casts checked instead of raw; missing patches metadata rejected; varbin merge loop boundary-wrapped. - SparseArrays.walkPatches: rejects non-ascending patch indices, which previously desynced the fill/patch cursor and over-emitted callbacks past the array's own length. - ChunkedEncodingDecoder: chunk offsets validated non-negative, non-decreasing, and spanning exactly rowCount; zero chunks with nonzero rowCount rejected uniformly across dtypes. - StructEncodingDecoder: added test-only coverage (guards already existed at the DType/PostscriptParser layer). - PrimitiveEncodingDecoder: reject a non-primitive requested dtype instead of an unchecked cast (a child can be decoded under a dtype not its own). TODO.md: mark these encodings done, seven of eleven per-encoding gotchas now closed.
dfa1
force-pushed
the
security/per-encoding-adversarial-tests
branch
from
August 5, 2026 19:54
78e32b7 to
3e587d9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of TODO.md's "Per-encoding adversarial tests" security item (CLAUDE.md §Security contract: a malformed file must always throw
VortexException, never a raw JDK exception).DecodeContext.VarBinArray's lazy accessors andSparseArrays.walkPatches) against crafted metadata that decodes but disagrees with the buffer payload — corrupted offsets, out-of-range indices/exponents/patch counts, zero-byte children, non-monotonic offsets, unsorted patch indices, and dtype/encoding mismatches that previously leakedArrayIndexOutOfBoundsException,IndexOutOfBoundsException,NegativeArraySizeException,ArithmeticException,ClassCastException,NullPointerException, orOutOfMemoryError.PrimitiveEncodingDecoder(rawClassCastExceptionon a dtype/encoding mismatch), found while hardening Sparse.TODO.md: 7 of 11 per-encoding gotchas closed; RLE/RunEnd, Constant, Zoned, Pco remain (follow-up PR).No wire-format or public-API changes — every fix is a decode-time validation guard or an accessor-level bounds check.
Test plan
./mvnw test -pl reader -am— reader/core/fsst unit tests green./mvnw verify -pl reader -am -DskipITs— includes checkstyle, green./mvnw verify(full 16-module reactor incl. failsafe integration tests against Rust-written files) — greenVortexExceptionfor every hardened case, added as@Nestedclasses in each encoding's existing decoder test file🤖 Generated with Claude Code