Fix AxiStreamDma stop races and zero-copy lifetime#1277
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adjusts rogue::hardware::axi::AxiStreamDma teardown semantics for zero-copy so that stop() closes only the per-instance DMA file descriptor while keeping the shared mapped DMA buffers alive until ~AxiStreamDma(), preventing downstream zero-copy frames retained past stop() from referencing unmapped memory.
Changes:
- Move shared descriptor (
desc_) cleanup fromstop()into~AxiStreamDma(). - Add
fdMtx_and use it to serialize deferreddmaRetIndex()calls againstfd_closure. - Remove unused locals in
retBuffer()and updatestop()/API comments to reflect the new lifetime model.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/rogue/hardware/axi/AxiStreamDma.cpp | Moves shared mapping cleanup to destructor; adds mutex-guarded fd close and mutex-guarded zero-copy buffer return. |
| include/rogue/hardware/axi/AxiStreamDma.h | Adds <mutex>, introduces fdMtx_, and updates stop() documentation to reflect new lifetime behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…fetime Add a Linux-only C++ doctest that constructs a real AxiStreamDma against an LD_PRELOAD shim faking the aes-stream-driver, retains a zero-copy frame across stop(), and asserts the shared DMA mapping stays mapped after stop() and is released only at destruction. Moving closeShared() back into stop() unmaps the buffers while the frame is still held and fails the post-stop() assertion, so the test guards against regressing the teardown-lifetime fix. The fake-driver shim is a test-only LD_PRELOAD module built only under ROGUE_BUILD_TESTS and never linked into rogue-core. The test is labeled no-python so it runs in both CI ctest jobs, and is skipped on non-Linux where LD_PRELOAD interposition is unavailable.
…clab/rogue into axi-stream-dma-stop-lifetime # Conflicts: # tests/cpp/hardware/CMakeLists.txt
doctest cannot stringify a shared_ptr<Frame>, so a bare REQUIRE(frame) fails to compile. Use static_cast<bool>(frame), matching the workaround already used in the sibling zerocopy lifetime test. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
I have a few concerns with this. I guess which is the error this is really solving. Since the fd_ value is only written twice and open and close and all other blocks read, I would suspect we would be fine without a lock. Were there actual cases where we were seeing segfaults on close because accept_frame was called? Also, I can understand the issue with the allocated buffer returns; that seems like a more real problem. Anyway, we should fully understand what is being solved. My main problem is the solution creates a huge amount of object creation and destruction to implement the lock. I would have preferred a more direct, but maybe less "layered and python" like approach that limited the memory allocations at high rate. The target high rate is also why I am apprehensive about even adding locks so I want to make sure this is really necessary. If we do have to add locks around the fd access, we need to use a clean mutex without the fancy class that wraps it. |
…clab/rogue into axi-stream-dma-stop-lifetime
|
I have reworked the change locally to focus on two cases:
The The public documentation now requires callers to quiesce stream operations and driver controls/accessors before calling The regression coverage has also been narrowed to the two behaviors above: retained mapping lifetime and |
Description
Fix
AxiStreamDmashutdown sostop()is safe while stream and driver operations are still in flight. The previous code could closefd_while another thread was inside a driver call such as zero-copy allocation, transmit, buffer return, or a getter. This is a use-after-close race on an OS file descriptor and can also allow the fd number to be reused for an unrelated resource.This also keeps the shared zero-copy DMA mapping alive until
~AxiStreamDma(), so retained Rogue frames do not point at unmapped DMA memory after a user callsstop().Details
stopped_state so new stream work is rejected once shutdown begins, while post-stop getters return safe default values.fd_guard backed bystd::shared_timed_mutex: driver operations take a shared lock, andstop()takes the exclusive lock before closing the descriptor. This guaranteesclose(fd_)cannot run while a guarded driver call is using that fd.acceptReq(),acceptFrame(),retBuffer(), driver getters, debug setup, and DMA ack.stop()to the destructor so zero-copy buffers retained afterstop()remain backed by valid mapped memory.stop()idempotent and join the RX thread based onjoinable().LD_PRELOADfake aes-stream-driver shim that blocks selected driver calls to deterministically provestop()waits instead of closing during active calls.Validation:
NO_PYTHONC++ hardware tests pass for retained zero-copy lifetime and stop-vs-driver-call races.AxiStreamDmachanges while keeping the new tests fails as expected: mappings are released atstop(), blocked calls can overlap descriptor close, and the fake shim detects close-during-driver-call.git diff --check,cmake --build build --target rogue-core-shared. The new hardware tests are Linux-only and are not registered on macOS.JIRA
N/A
Related
Related to #1276 teardown review.