Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
c7dbf6d
Add incremental Racer workspace CI for stacked PRs
jveski Oct 5, 2026
c0adfdd
Update CI workflow to exclude 'pr/racer-*' branches
jveski Oct 5, 2026
e0c6e80
feat(racer): import page allocator crate
jveski Oct 5, 2026
9dd6b06
ci(racer): validate isolated allocator production and simulation
jveski Oct 5, 2026
1064d9b
fix(racer): honor segment eviction veto in allocator clock
jveski Oct 5, 2026
12bfce1
refactor(racer): tighten allocator helper visibility and spacing
jveski Oct 5, 2026
39f84b8
fix(alloc): track clean storage and avoid redundant secure wipes
jveski Oct 5, 2026
cac312b
test(alloc): verify pooled erasure across read completion fences
jveski Oct 5, 2026
45499b1
Refactor registration handling in slab.rs
jveski Oct 6, 2026
6f6ad84
fix(alloc): replace registered waker through RefCell
Copilot Oct 6, 2026
cc5c113
test(alloc): validate alignment without kernel fallback assumptions
jveski Oct 6, 2026
017b216
fix(alloc): wake slab fences outside registration borrows
jveski Oct 6, 2026
fb63bb8
docs(racer): add page allocator design
jveski Oct 7, 2026
fc3d83c
docs(racer): drop open questions from page allocator design
jveski Oct 7, 2026
1e5f44a
feat(racer): extract flow-control library
jveski Oct 5, 2026
97122ca
test(racer): gate isolated flow production and simulation
jveski Oct 5, 2026
975176b
fix(racer): preserve flow quotas when admission key cloning panics
jveski Oct 5, 2026
f294253
fix(racer): retain flow ownership through panic and cancellation
jveski Oct 5, 2026
2f439b2
refactor(racer): consolidate flow integration workflows
jveski Oct 5, 2026
7a5fbf7
Fix flow control review findings
Copilot Oct 6, 2026
ba4aa2d
Simplify recycler test type
Copilot Oct 6, 2026
360c0eb
feat(page-alloc): add multi-placement device backing
jveski Oct 7, 2026
e5889f7
Merge port/racer-block-alloc into port/racer-block-flow
jveski Oct 7, 2026
5d23a0d
test(gantry): synchronize late provider with cold-start entry
jveski Oct 7, 2026
097bcd6
fix(flow): keep cohort completion results immutable
jveski Oct 7, 2026
6d1f7ed
fix(flow): recheck stop after shared quota reservation
jveski Oct 7, 2026
7146be8
fix(flow): keep adaptive key clone panics outside admission lock
jveski Oct 7, 2026
fed1144
fix(flow): release key borrow before rejection callback
jveski Oct 7, 2026
445d2d6
fix(flow): allow reentrant circuit backoff callbacks
jveski Oct 7, 2026
adb5bc5
Merge origin/main into pr/racer-alloc
jveski Oct 7, 2026
884b848
Merge origin/main into pr/racer-alloc
jveski Oct 7, 2026
eab17a3
test(alloc): derive device placement sizes from alignment
jveski Oct 7, 2026
cea65e6
docs(alloc): clarify device-backed startup ownership
jveski Oct 7, 2026
84cd3f7
fix(flow): recheck local admission stop after callbacks
jveski Oct 7, 2026
a2bce7f
fix(flow): enforce pipe waiter limit across reentry
jveski Oct 7, 2026
b8d033d
Merge pr/racer-alloc into pr/racer-flow
jveski Oct 7, 2026
bbdb46d
fix(flow): preserve handoff admission errors
jveski Oct 7, 2026
8aafc70
fix(flow): avoid closed descriptor probes in pipe test
jveski Oct 7, 2026
bd03958
fix(flow): preserve pressure eligibility at recovery ceiling
jveski Oct 7, 2026
ebc4b84
fix(flow): clear completed hedge delay wake registration
jveski Oct 7, 2026
8147019
fix(flow): handle admission duration overflow
jveski Oct 7, 2026
8591ad5
Merge main into pr/racer-alloc
jveski Oct 7, 2026
e911b1b
fix(alloc): recheck write fence after unregister callbacks
jveski Oct 7, 2026
8f0365f
Merge pr/racer-alloc into pr865 merge-ready
jveski Oct 7, 2026
a302c51
fix(alloc): drain pending segment reclamation victims
jveski Oct 7, 2026
0f6474f
fix(alloc): wait for scored reclamation to drain
jveski Oct 7, 2026
1349aaa
test(alloc): use American English cancellation spelling
jveski Oct 7, 2026
9edad30
docs(alloc): clarify worker-local allocation authority
jveski Oct 7, 2026
2abb49b
docs(alloc): describe exact-size idle buffer reuse
jveski Oct 7, 2026
cdb5969
docs(alloc): use American English cancellation spelling
jveski Oct 7, 2026
da5042f
fix(flow): retire keyed charge before waking admission
jveski Oct 7, 2026
e505f01
fix(alloc): reject restored generation rollback
jveski Oct 7, 2026
b1bee69
test(alloc): honor optional direct I/O capability skips
jveski Oct 7, 2026
dc60564
fix(flow): clone recycler keys before locking
jveski Oct 7, 2026
652fad8
fix(flow): revalidate dynamic limits before reclamation
jveski Oct 7, 2026
c7d3e99
fix(flow): deduplicate handoff target keys
jveski Oct 7, 2026
af87cf3
fix(flow): release reclamation key borrow before callbacks
jveski Oct 7, 2026
4f62bac
fix(alloc): reject oversized startup tables before opening files
jveski Oct 7, 2026
4ba02db
fix(alloc): check block-device placement capacity
jveski Oct 7, 2026
da579f5
test(alloc): exercise aligned device overlaps and inode aliases
jveski Oct 7, 2026
6bbbc52
test(alloc): probe alignment before sizing real I/O workflows
jveski Oct 7, 2026
3f90a04
docs(alloc): clarify pending eviction reclaim results
jveski Oct 7, 2026
37623ed
docs(alloc): state trusted caller write contract
jveski Oct 7, 2026
50363e2
docs(alloc): distinguish leases from initialized records
jveski Oct 7, 2026
68e957c
docs(alloc): describe scored reclamation visit limit
jveski Oct 7, 2026
7257f69
docs(alloc): replace stale design line references
jveski Oct 7, 2026
253358a
fix(flow): track peer idle age separately from recovery
jveski Oct 7, 2026
253af45
fix(flow): release admission and wake waiters on handoff close
jveski Oct 7, 2026
27b8202
test(alloc): derive startup geometry from probed alignment
jveski Oct 7, 2026
968dff0
fix(flow): fence exhausted adaptive generations
jveski Oct 7, 2026
4f3fb37
fix(flow): use state-neutral unavailable diagnostic
jveski Oct 7, 2026
3ba6998
Merge current pr/racer-alloc into pr865 merge-ready
jveski Oct 7, 2026
8855495
fix(page-alloc): revalidate file metadata under lock
jveski Oct 7, 2026
1caf390
test(page-alloc): derive lock handoff geometry from probe
jveski Oct 7, 2026
8617658
fix(flow): run handoff waker callbacks outside the lock
jveski Oct 7, 2026
15e9365
fix(flow): stage hedge delay wakers outside lock
jveski Oct 7, 2026
78583c5
fix(flow): drop hedge alarm wakers outside the lock
jveski Oct 7, 2026
e37aef4
fix(flow): release cohort borrows before waker callbacks
jveski Oct 7, 2026
9878035
fix(flow): drop shared receiver outside table borrow
jveski Oct 7, 2026
4f819c2
fix(flow): stage drain wakers outside owner borrows
jveski Oct 7, 2026
4e2931f
fix(flow): defer removed entry destruction past owner borrows
jveski Oct 7, 2026
d0cb519
fix(flow): release pipe waiter borrows before waker callbacks
jveski Oct 7, 2026
c0b2a41
fix(flow): reject occupied flight insertion
jveski Oct 7, 2026
3016e0a
fix(flow): clone coalesce keys outside table borrows
jveski Oct 7, 2026
b609301
fix(flow): drop retired adaptive keys after unlocking
jveski Oct 7, 2026
587e513
fix(flow): join shared flights admitted during key cloning
jveski Oct 7, 2026
18cfd77
fix(flow): retire flight wakers outside owner borrows
jveski Oct 7, 2026
a27f9da
fix(flow): clone quota keys before borrowing the map
jveski Oct 7, 2026
3e7cb87
fix(flow): drain idle pipes when admission observes stop
jveski Oct 7, 2026
cf5fee3
Merge pr/racer-alloc into fix/pr865-merge-ready
jveski Oct 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,44 @@ jobs:
RUNTIME_REQUIRE_IO_URING: "1"
run: timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 test --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p uring-runtime --no-default-features

# Keep allocator invocations separate from runtime/workspace tests: runtime
# test features must not enable simulation in the production allocator gate.
- name: Check allocator production and simulation
run: |
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 check --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p page-alloc --all-targets --no-default-features
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 check --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p page-alloc --all-targets --no-default-features --features simulation

- name: Strict allocator Clippy
run: |
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 clippy --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p page-alloc --all-targets --no-default-features -- -D warnings
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 clippy --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p page-alloc --all-targets --no-default-features --features simulation -- -D warnings

- name: Allocator production real I/O (no simulation)
env:
PAGE_ALLOC_REQUIRE_REAL_IO: "1"
run: timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 test --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p page-alloc --no-default-features

- name: Test allocator simulation
run: timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 test --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p page-alloc --no-default-features --features simulation

# Isolate flow features so workspace tests cannot substitute simulated I/O
# for the production pipe and socket contracts. Real-I/O failures never skip.
- name: Check flow production and simulation
run: |
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 check --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p flow-control --all-targets --no-default-features
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 check --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p flow-control --all-targets --no-default-features --features simulation

- name: Strict flow Clippy
run: |
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 clippy --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p flow-control --all-targets --no-default-features -- -D warnings
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 clippy --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p flow-control --all-targets --no-default-features --features simulation -- -D warnings

- name: Flow production real I/O (no simulation)
run: timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 test --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p flow-control --no-default-features

- name: Test flow simulation
run: timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 test --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p flow-control --no-default-features --features simulation

racer-verbs:
name: Racer Verbs (${{ matrix.configuration }})
runs-on: ubuntu-24.04
Expand Down
20 changes: 20 additions & 0 deletions cmd/racer-dataplane/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion cmd/racer-dataplane/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[workspace]
members = [".", "topology", "runtime", "telemetry", "verbs"]
members = [".", "topology", "runtime", "alloc", "flow", "telemetry", "verbs"]
resolver = "3"

[package]
Expand Down
35 changes: 35 additions & 0 deletions cmd/racer-dataplane/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,38 @@ timeout --signal=TERM --kill-after=10s 300s python3 hack/scripts/runtime-miri.py

Run the `offload`, `scheduler`, and `memory` groups separately with the same script.
Each group verifies that every exact allowlisted test actually ran.

The `page-alloc` library provides worker-local aligned buffers, lease-fenced slab
I/O, and bounded segment reclamation. Test it separately from runtime/workspace
tests so feature unification cannot enable simulation in its production gate:

```sh
PAGE_ALLOC_REQUIRE_REAL_IO=1 timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 test --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p page-alloc --no-default-features -j 2 -- --test-threads=1
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 test --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p page-alloc --no-default-features --features simulation -j 2 -- --test-threads=1
```

The production gate requires real io_uring and direct-I/O support: capability
skips fail when `PAGE_ALLOC_REQUIRE_REAL_IO=1`. CI also checks all targets and
runs strict Clippy for each allocator mode.

The `flow-control` library provides policy-driven quotas, charged buffers, bounded
kernel pipes, adaptive admission, and worker-local coalescing. Application policy
and real operation completion remain the caller's responsibility. Its crate docs
describe ownership and admission contracts; generate them with:

```sh
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 doc --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p flow-control --no-deps
```

Validate flow separately so workspace feature unification cannot enable simulation
in its production gate:

```sh
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 test --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p flow-control --no-default-features -j 2 -- --test-threads=1
timeout --signal=TERM --kill-after=10s 300s cargo +1.96.0 test --locked --manifest-path cmd/racer-dataplane/Cargo.toml -p flow-control --no-default-features --features simulation -j 2 -- --test-threads=1
```

The flow production tests require Linux kernel pipes, Unix sockets, and splice.
They exercise real I/O and fail rather than skipping unsupported operations.
Simulation adds simulated and mixed-descriptor contracts without removing the
real pipe tests. CI checks all targets and runs strict Clippy in each mode.
1 change: 1 addition & 0 deletions cmd/racer-dataplane/alloc/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/target/
16 changes: 16 additions & 0 deletions cmd/racer-dataplane/alloc/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[package]
name = "page-alloc"
version = "0.1.0"
edition = "2024"
license = "Apache-2.0"
publish = false
description = "Worker-local aligned buffers and lease-fenced sparse slab storage"

[features]
default = []
simulation = ["uring-runtime/simulation"]

[dependencies]
libc = "0.2"
zeroize = "1"
uring-runtime = { path = "../runtime" }
Loading
Loading