Skip to content

Validate packet burst bounds before writes#202

Open
fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix/packet-bounds-validation
Open

Validate packet burst bounds before writes#202
fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix/packet-bounds-validation

Conversation

@fallintoplace

Copy link
Copy Markdown
Contributor

Summary

  • add a shared burst validation helper for packet counts, segment indexes, packet indexes, payload lengths, packet lengths, and backing capacities
  • guard public burst accessors, payload setters, packet-length setters, TX burst acquisition, send, and release paths across DPDK, socket, RDMA, and ibverbs engines
  • reject oversized or malformed packet metadata before memcpy, length assignment, or segment access

Testing

  • git diff --check
  • cmake -S . -B build-verify -G Ninja -DDAQIRI_ENGINE="" -DDAQIRI_BUILD_EXAMPLES=OFF -DDAQIRI_BUILD_PYTHON=OFF (fails locally before targets: CUDA toolkit/nvcc is not installed)

@greptile-apps

greptile-apps Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR centralizes burst bounds validation across all four engines (DPDK, ibverbs, RDMA, socket) into a new src/burst_validation.h helper header, and adds a DAQIRI_ENABLE_BURST_VALIDATION CMake option (default ON) to allow the checks to be compiled out for trusted-input performance paths.

  • New burst_validation.h provides typed helpers (validate_packet_index, validate_segment_count, validate_payload_write, etc.) that guard against negative indices, oversized segment counts, null payload sources, and backing-capacity overflows; used uniformly across all engine accessor, setter, TX acquisition, send, and free paths.
  • Socket cleanup paths adopt a cleanup_header_limits wrapper that clamps corrupt num_segs values before freeing, preserving the old std::clamp behaviour in free_all_packets.
  • RDMA replaces two assert() calls in get_tx_packet_burst with proper error returns, and ibverbs introduces per-queue slot-size capacity helpers (tx_segment_capacity, tx_segment_capacities) to bound-check payload writes against registered memory.

Confidence Score: 4/5

Safe to merge after fixing the ibverbs set_packet_lengths guard removal; all other engine paths are correct.

The ibverbs set_packet_lengths loop lost the seg >= MAX_NUM_SEGS guard that capped writes to burst->pkt_lens. When DAQIRI_ENABLE_BURST_VALIDATION=0 the replacement validation skips the lens.size() == num_segs check, so a caller passing more lengths than MAX_NUM_SEGS will overwrite memory beyond the array. Every other validation addition across the four engines is consistent and correct.

src/engines/ibverbs/daqiri_ibverbs_engine.cpp — the set_packet_lengths write loop after the validation call needs the seg >= MAX_NUM_SEGS guard restored or the non-strict path of validate_packet_lengths needs to enforce the size limit.

Important Files Changed

Filename Overview
src/burst_validation.h New shared validation header; logic is correct; minor readability nit in validate_burst null-check structure
src/engines/ibverbs/daqiri_ibverbs_engine.cpp Guard removal in set_packet_lengths creates an out-of-bounds pkt_lens write when validation is disabled; rest of the validation additions are correct
src/engines/dpdk/daqiri_dpdk_engine.cpp Validation guards added correctly throughout; ttl_len = packet_len assignment in set_all_packet_lengths validation loop is technically correct but reads like an unintended overwrite rather than accumulation
src/engines/socket/daqiri_socket_engine.cpp Cleanup paths correctly use clamped cleanup_header_limits to preserve segment iteration behavior; validation additions look consistent
src/engines/rdma/daqiri_rdma_engine.cpp Replaces assert guards with proper error returns; validates segment count before RDMA TX burst; set_packet_lengths adds MR capacity check correctly
CMakeLists.txt Adds DAQIRI_ENABLE_BURST_VALIDATION CMake option at the root level; correct default-ON
src/CMakeLists.txt Re-declares option and emits compile definition; README not updated per doc-sync rule
AGENTS.md Adds DAQIRI_ENABLE_BURST_VALIDATION to CMake options list and removes stale line-number reference
docs/getting-started.md Adds DAQIRI_ENABLE_BURST_VALIDATION row to the CMake options table
docs/tutorials/bare-metal-cmake-build.md Adds description of the new CMake option to the build tutorial
src/engines/ibverbs/daqiri_ibverbs_engine.h Declares three new TX segment capacity helpers; <cstddef> include added
src/engines/rdma/daqiri_rdma_engine.h Declares local_mr_capacity helper; <cstddef> include added
src/engines/socket/daqiri_socket_engine.h Declares packet_capacity helper; <cstddef> include added

Reviews (2): Last reviewed commit: "#202 - Validate packet burst bounds befo..." | Re-trigger Greptile

Comment on lines 411 to 436
}

void SocketEngine::free_all_packets(BurstParams* burst) {
if (burst == nullptr) { return; }
const int num_segs = std::clamp(burst->hdr.hdr.num_segs, 0, MAX_NUM_SEGS);
for (int seg = 0; seg < num_segs; ++seg) {
if (burst_validation::validate_segment_count(
burst,
burst_validation::header_limits(burst),
"SocketEngine::free_all_packets") != Status::SUCCESS) {
return;
}
for (int seg = 0; seg < burst->hdr.hdr.num_segs; ++seg) {
free_all_segment_packets(burst, seg);
}
}

void SocketEngine::free_packet_segment(BurstParams* burst, int seg, int pkt) {
if (burst == nullptr || seg < 0 || seg >= MAX_NUM_SEGS || burst->pkts[seg] == nullptr || pkt < 0 ||
pkt >= static_cast<int>(burst->hdr.hdr.num_pkts)) {
if (burst_validation::validate_segment_packet_storage(
burst,
burst_validation::header_limits(burst),
seg,
pkt,
true,
false,
"SocketEngine::free_packet_segment") != Status::SUCCESS) {
return;
}
auto* data = reinterpret_cast<uint8_t*>(burst->pkts[seg][pkt]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 free_all_packets no longer frees segments when num_segs > MAX_NUM_SEGS

The previous implementation used std::clamp(burst->hdr.hdr.num_segs, 0, MAX_NUM_SEGS) to iterate up to MAX_NUM_SEGS segments even when the stored count was out of range. The replacement validate_segment_count returns early (and skips all free_all_segment_packets calls) whenever num_segs <= 0 || num_segs > MAX_NUM_SEGS. The inner free_all_segment_packets now also validates via validate_segment_index, so the double-rejection is consistent; but any burst that arrives with a corrupt-but-positive segment count above the limit will now have none of its packet buffers freed. For the socket engine, which heap-allocates each packet with new[], those pointers are permanently lost. The num_segs > MAX_NUM_SEGS case is unlikely in normal operation but the behaviour change is silent.

@cliffburdick

Copy link
Copy Markdown
Collaborator

This PR appears to add a significant amount of checks that are per-packet or per-segment, which quickly has a large overhead for small packet sizes. Can you characterize the performance loss from having this, and if it's noticeable, put it behind an compilation flag?

Centralize packet and segment bounds checks for public burst accessors and setters across DPDK, socket, RDMA, and ibverbs engines.

Add DAQIRI_ENABLE_BURST_VALIDATION so strict burst validation remains on by default but can be disabled for trusted-input performance testing.

Preserve socket cleanup behavior by clamping corrupt segment counts before freeing packet buffers.

Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
@fallintoplace fallintoplace force-pushed the fix/packet-bounds-validation branch from 25067b3 to 74e21cc Compare June 30, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants