fix(reader): harden 7 encodings against malformed adversarial input

CHANGELOG.md

@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+
+- Malformed files no longer crash the reader with a raw JDK exception when decoding VarBin, Dict, Bitpacked, ALP, Sparse, Chunked, or Struct columns — every case now fails as `VortexException`. ([ef982992](https://github.com/dfa1/vortex-java/commit/ef982992))
+
 ### Added
 
 - `MemorySize` domain primitive in `core.model` replaces raw byte-count `long`/`int` arithmetic: validated non-negative at construction, with `ofKiB`/`ofMiB`/`ofGiB` factories and a `toGiB()` display accessor. Migrated `WriteOptions#globalDictMaxRetainedBytes()`, `PostscriptParser`'s layout-metadata size cap, `DType.Extension#MAX_METADATA_SIZE`, and `VortexHttpReader`'s HTTP tail-fetch window. ([#321](https://github.com/dfa1/vortex-java/issues/321))

TODO.md

@@ -35,20 +35,11 @@ known gap, a contract audit, or supporting infra.
 
 ### Per-encoding adversarial tests
 
-Each encoding's `decode(DecodeContext)` should be exercised against:
-- `bufferIndices[i] >= ctx.bufferCount()` → centralize check in `DecodeContext.buffer(i)`.
-- Crafted metadata that decodes but disagrees with the buffer payload.
-
-Per-encoding gotchas:
-- [ ] **VarBin**: offsets non-monotonic, negative, past data-buffer length.
-- [ ] **Dict**: `codes[i] >= values.length`; `codes` ptype declared u8 but values count > 256.
-- [ ] **Bitpacked**: `bit_width < 0 || > 64`; `packed_len < n * bit_width / 8`.
-- [ ] **ALP**: `dim < 0`, `f_or_d` byte out of enum range; `exceptions_count > n`.
-- [ ] **Sparse**: indices non-sorted or `indices[i] >= length`; values count
-  mismatches indices count.
-- [ ] **Chunked**: zero children with non-zero `row_count`; child layout self-referencing
-  (already protected by depth limit, but add explicit test).
-- [ ] **Struct**: `fieldNames.size() != children.size()`; field name UTF-8 invalid.
+Each encoding's `decode(DecodeContext)` should be exercised against crafted metadata that
+decodes but disagrees with the buffer payload. `bufferIndices[i] >= ctx.bufferCount()` (and the
+equivalent child-index check) is centralized in `DecodeContext.buffer(i)`/`decodeChild(i)`.
+VarBin, Dict, Bitpacked, ALP, Sparse, Chunked, and Struct are done — remaining gotchas:
+
 - [ ] **RLE / RunEnd**: `run_ends` non-monotonic; last `run_end` ≠ `row_count`.
 - [ ] **Constant**: protobuf scalar value missing or type-mismatched against declared `DType`.
 - [ ] **Zoned**: zone-map min > max; zone count ≠ child chunk count.

reader/src/main/java/io/github/dfa1/vortex/reader/array/SparseArrays.java

@@ -89,6 +89,18 @@ static void walkPatches(Array patchIndices, long numPatches, long absStart, long
             if (patchAbs >= absEnd) {
                 break;
             }
+            // patchIndices is a wire-supplied, untrusted array that the format requires to be
+            // sorted ascending: findFirstAtOrAfter's binary search assumes it, and this walk
+            // advances `pos` to `patchAbs + 1` on each patch. A patch that goes backwards moves
+            // `pos` backwards too, so a later fill/patch run re-covers already-emitted
+            // positions — the walk then emits more callbacks than `absEnd - absStart`, which
+            // overflows a caller's fixed-size output buffer as a raw IndexOutOfBoundsException
+            // rather than a VortexException (ADR 0003). Checked once per patch, not per row.
+            if (patchAbs < pos) {
+                throw new VortexException(
+                        "Sparse patch indices not sorted: index " + patchAbs + " at patch " + p
+                                + " precedes position " + pos);
+            }
             for (long r = pos; r < patchAbs; r++) {
                 fillSlot.run();
             }

reader/src/main/java/io/github/dfa1/vortex/reader/decode/AlpEncodingDecoder.java

@@ -82,13 +82,15 @@ public Array decode(DecodeContext ctx) {
 
     private static Array decodeF64(DecodeContext ctx, ProtoALPMetadata meta, int expE, int expF, long n,
             MemorySegment src) {
+        checkExponents(expE, expF, F10_F64.length);
         // Decode formula mirrors the Rust reference (`ALPFloat::decode_single`): two-step
         // `encoded * F10[f] * IF10[e]`. A pre-multiplied `scale = F10[f] * IF10[e]`
         // gives different IEEE rounding for non-trivial `expF`, breaking round-trip with
         // the encoder's verify step.
         double df = F10_F64[expF];
         double de = IF10_F64[expE];
         long srcCap = SegmentBroadcast.capacity(src, 8);
+        checkSource(srcCap, n);
 
         if (meta.patches() == null) {
             if (srcCap >= n) {
@@ -116,9 +118,11 @@ private static Array decodeF64(DecodeContext ctx, ProtoALPMetadata meta, int exp
 
     private static Array decodeF32(DecodeContext ctx, ProtoALPMetadata meta, int expE, int expF, long n,
             MemorySegment src) {
+        checkExponents(expE, expF, F10_F32.length);
         float df = F10_F32[expF];
         float de = IF10_F32[expE];
         long srcCap = SegmentBroadcast.capacity(src, 4);
+        checkSource(srcCap, n);
 
         if (meta.patches() == null) {
             if (srcCap >= n) {
@@ -144,30 +148,88 @@ private static Array decodeF32(DecodeContext ctx, ProtoALPMetadata meta, int exp
         return new MaterializedFloatArray(ctx.dtype(), n, buf.asReadOnly());
     }
 
+    /// Rejects out-of-range ALP exponents before they index the power-of-ten tables.
+    ///
+    /// `exp_e` and `exp_f` are untrusted metadata read verbatim from the file; a negative or
+    /// oversized exponent used to escape as an `ArrayIndexOutOfBoundsException` from the
+    /// table lookup rather than a [VortexException] (ADR 0003). The check is O(1) and runs
+    /// once per decode, outside every loop.
+    ///
+    /// @param expE   the `e` exponent, an index into the inverse power-of-ten table
+    /// @param expF   the `f` exponent, an index into the power-of-ten table
+    /// @param maxExp exclusive upper bound — the table length for this float width
+    private static void checkExponents(int expE, int expF, int maxExp) {
+        if (expE < 0 || expE >= maxExp || expF < 0 || expF >= maxExp) {
+            throw new VortexException(EncodingId.VORTEX_ALP,
+                    "exponents (e=" + expE + ", f=" + expF + ") out of range [0," + maxExp + ")");
+        }
+    }
+
+    /// Rejects an encoded child that carries no element at all.
+    ///
+    /// Every path below reads at least element 0 — the broadcast path reads exactly it, and
+    /// the row-wise paths wrap with `% srcCap` — so a zero-length child would either read
+    /// off the segment or divide by zero. Checked once, outside the row loops.
+    ///
+    /// @param srcCap number of elements physically present in the encoded child
+    /// @param n      logical row count
+    private static void checkSource(long srcCap, long n) {
+        if (srcCap == 0 && n > 0) {
+            throw new VortexException(EncodingId.VORTEX_ALP,
+                    "empty encoded child for " + n + " rows");
+        }
+    }
+
     private static void applyPatches(DecodeContext ctx, ProtoPatchesMetadata pm, MemorySegment out, int elemBytes) {
         long numPatches = pm.len();
+        if (numPatches == 0) {
+            return;
+        }
         long offset = pm.offset();
         PType idxPtype = PType.fromOrdinal(pm.indices_ptype().value());
         int idxBytes = idxPtype.byteSize();
+        long n = out.byteSize() / elemBytes;
 
         MemorySegment idxSeg = ctx.decodeChildSegment(1, new DType.Primitive(idxPtype, false), numPatches);
         MemorySegment valSeg = ctx.decodeChildSegment(2, ctx.dtype(), numPatches);
 
         long idxCap = SegmentBroadcast.capacity(idxSeg, idxBytes);
         long valCap = SegmentBroadcast.capacity(valSeg, elemBytes);
+        if (idxCap == 0 || valCap == 0) {
+            throw new VortexException(EncodingId.VORTEX_ALP,
+                    "empty patch child for " + numPatches + " declared patches (indices="
+                            + idxSeg.byteSize() + " bytes, values=" + valSeg.byteSize() + " bytes)");
+        }
         if (idxCap >= numPatches && valCap >= numPatches) {
             for (long i = 0; i < numPatches; i++) {
                 long absIdx = readUnsigned(idxSeg, i * idxBytes, idxPtype) - offset;
+                checkPatchIndex(absIdx, n);
                 MemorySegment.copy(valSeg, i * elemBytes, out, absIdx * elemBytes, elemBytes);
             }
         } else {
             for (long i = 0; i < numPatches; i++) {
                 long absIdx = readUnsigned(idxSeg, (i % idxCap) * idxBytes, idxPtype) - offset;
+                checkPatchIndex(absIdx, n);
                 MemorySegment.copy(valSeg, (i % valCap) * elemBytes, out, absIdx * elemBytes, elemBytes);
             }
         }
     }
 
+    /// Guards a patch scatter-write against an untrusted index, mirroring the identical
+    /// check in [BitpackedEncodingDecoder]: a patch index outside the row range (or one
+    /// pushed negative by `patches.offset`) must fail as a [VortexException] rather than as
+    /// a raw `IndexOutOfBoundsException` from the copy. Patches are sparse, so the per-patch
+    /// test costs nothing on the row-wise decode loops.
+    ///
+    /// @param absIdx absolute row index of the patch, after subtracting `patches.offset`
+    /// @param n      logical row count
+    private static void checkPatchIndex(long absIdx, long n) {
+        if (absIdx < 0 || absIdx >= n) {
+            throw new VortexException(EncodingId.VORTEX_ALP,
+                    "patch index " + absIdx + " out of range [0," + n + ")");
+        }
+    }
+
     private static long readUnsigned(MemorySegment seg, long off, PType ptype) {
         return switch (ptype) {
             case U8 -> Byte.toUnsignedLong(seg.get(ValueLayout.JAVA_BYTE, off));

reader/src/main/java/io/github/dfa1/vortex/reader/decode/BitpackedEncodingDecoder.java

@@ -23,6 +23,10 @@
 public final class BitpackedEncodingDecoder implements EncodingDecoder {
     private static final int[] FL_ORDER = {0, 4, 2, 6, 1, 5, 3, 7};
 
+    /// Elements per FastLanes block; also the exclusive upper bound of the in-block
+    /// `offset` carried in the metadata.
+    private static final int FL_BLOCK_SIZE = 1024;
+
     @Override
     public EncodingId encodingId() {
         return EncodingId.FASTLANES_BITPACKED;
@@ -52,9 +56,34 @@ public Array decode(DecodeContext ctx) {
         int typeBits = ptype.bits();
         long rowCount = ctx.rowCount();
 
+        // bit_width and offset come straight off the wire and drive every byte offset the
+        // unpack loops compute, so both are range-checked up front (O(1), outside any
+        // loop). The width bound is the column's own element width, not a flat 64: a
+        // 40-bit width on an I8 column is just as malformed as a 65-bit one. The offset is
+        // a position inside the first 1024-element FastLanes block, and an oversized one
+        // would spin through billions of skipped lanes before failing.
+        if (bitWidth < 0 || bitWidth > typeBits) {
+            throw new VortexException(EncodingId.FASTLANES_BITPACKED,
+                    "bit width " + bitWidth + " out of range [0," + typeBits + "] for " + ptype);
+        }
+        if (offset < 0 || offset >= FL_BLOCK_SIZE) {
+            throw new VortexException(EncodingId.FASTLANES_BITPACKED,
+                    "offset " + offset + " out of range [0," + FL_BLOCK_SIZE + ")");
+        }
+
         MemorySegment packed = ctx.buffer(0);
         MemorySegment output = ctx.arena().allocate(rowCount * ptype.byteSize());
-        fastlanesUnpackToSeg(packed, bitWidth, offset, typeBits, rowCount, output);
+        // A packed buffer shorter than the block/lane math requires overruns `packed`.
+        // Computing the exact required length here would duplicate that math for every
+        // width, so the overrun is caught at the call boundary and reported as a
+        // VortexException instead of a raw IndexOutOfBoundsException (ADR 0003).
+        try {
+            fastlanesUnpackToSeg(packed, bitWidth, offset, typeBits, rowCount, output);
+        } catch (IndexOutOfBoundsException e) {
+            throw new VortexException(EncodingId.FASTLANES_BITPACKED,
+                    "packed buffer of " + packed.byteSize() + " bytes is too small for "
+                            + rowCount + " rows at bit width " + bitWidth, e);
+        }
 
         if (meta.patches() != null) {
             applyPatches(ctx, meta.patches(), output, ptype.byteSize());

reader/src/main/java/io/github/dfa1/vortex/reader/decode/ChunkedEncodingDecoder.java

@@ -36,7 +36,16 @@ public Array decode(DecodeContext ctx) {
                     "needs at least one child (chunk offsets)");
         }
         int nchunks = nchildren - 1;
-        long[] offsets = readOffsets(ctx, nchunks);
+        // A chunked node with only the offsets child carries no data at all, so it can never
+        // describe a non-empty column. Rejecting it here names the real defect uniformly;
+        // letting it through would surface as the less direct `Chunked*Array` "empty chunk
+        // list" error for most dtypes, and as a silently empty zero-field struct for
+        // DType.Struct (any struct with a field routes through the same "empty chunk list").
+        if (nchunks == 0 && ctx.rowCount() > 0) {
+            throw new VortexException(EncodingId.VORTEX_CHUNKED,
+                    "no chunks for " + ctx.rowCount() + " row(s)");
+        }
+        long[] offsets = readOffsets(ctx, nchunks, ctx.rowCount());
 
         DType dtype = ctx.dtype();
         List<Array> chunks = new ArrayList<>(nchunks);
@@ -48,12 +57,40 @@ public Array decode(DecodeContext ctx) {
         return wrap(chunks, dtype, ctx.rowCount());
     }
 
-    private static long[] readOffsets(DecodeContext ctx, int nchunks) {
+    /// Reads the `nchunks + 1` cumulative chunk offsets and validates them.
+    ///
+    /// The offsets buffer is untrusted: an empty one would make the broadcast wrap divide by
+    /// zero, and a non-monotonic or negative pair yields a negative chunk length that flows
+    /// into the child decode and then into a `Chunked*Array` whose `offsets` are no longer
+    /// sorted — making its binary-search dispatch index a chunk at a negative row. Both must
+    /// fail as a [VortexException], never as an `ArithmeticException` or a raw
+    /// `IndexOutOfBoundsException` (ADR 0003). The scan is O(nchunks), not per row.
+    ///
+    /// The final offset is also checked against `rowCount`: without this, an offsets pair like
+    /// `[0, 1 << 40]` for a 4-row array passes the monotonic check yet hands a terabyte-sized
+    /// chunk length to `decodeChild`, which allocates it before anything downstream can reject
+    /// it — an `OutOfMemoryError` rather than a `VortexException` (ADR 0003).
+    ///
+    /// @param ctx      decode context
+    /// @param nchunks  number of data chunks (children minus the offsets child)
+    /// @param rowCount logical row count the chunk offsets must sum to
+    /// @return the `nchunks + 1` monotonic, non-negative offsets spanning exactly `rowCount`
+    private static long[] readOffsets(DecodeContext ctx, int nchunks, long rowCount) {
         MemorySegment offsetsBuf = ctx.decodeChildSegment(0, DType.U64, nchunks + 1L);
-        long cap = SegmentBroadcast.capacity(offsetsBuf, 8);
         long[] offsets = new long[nchunks + 1];
         for (int i = 0; i <= nchunks; i++) {
-            offsets[i] = offsetsBuf.get(LE_LONG, (i % cap) * 8);
+            offsets[i] = offsetsBuf.get(LE_LONG, SegmentBroadcast.elementOffset(offsetsBuf, i, 8));
+            long previous = i == 0 ? 0 : offsets[i - 1];
+            if (offsets[i] < previous) {
+                throw new VortexException(EncodingId.VORTEX_CHUNKED,
+                        "chunk offsets must be non-negative and non-decreasing, got offsets["
+                                + i + "]=" + offsets[i] + " after " + previous);
+            }
+        }
+        long span = offsets[nchunks] - offsets[0];
+        if (span != rowCount) {
+            throw new VortexException(EncodingId.VORTEX_CHUNKED,
+                    "chunk offsets span " + span + " rows, expected " + rowCount);
         }
         return offsets;
     }