Skip to content

kernel: no NUMA topology, so memory and tasks are placed without regard to node #516

Open
2 of 3 issues completed

Description

@lexoliu

Helios has no notion of memory locality. No backend reads a NUMA description (ACPI SRAT and SLIT, or the device tree's numa-node-id and distance-map): the firmware tables parsed today are MADT and SPCR. hal hands the kernel memory as a flat list of regions (hal::memory::MemoryRegion) and processors as a table with no node. The frame allocator, the per-processor frame cache (kernel/src/memory/frame_slab.rs), the pooling allocator's linear-memory slots, fiber stacks and the executor's work stealing all treat every frame and every processor as equidistant.

On a single-node guest that is exact. On a multi-node one (a large cloud instance type, a multi-socket host) an instance's memory lands on whichever node the allocator reaches first and its task migrates across nodes freely, where Linux under the same Wasmtime allocates first-touch on the running processor's node and keeps a task near its memory. That is a parity loss (#311) on every memory-bound workload, and one no current lane can see: every lane boots a single-node guest.

Scope

  • Contract (hal). A memory region and a processor each carry a node, and the platform publishes the node distance matrix. A platform with no NUMA description is one node, with no special case in the kernel.
  • Backends. x86-64 and aarch64 read SRAT and SLIT; aarch64 and riscv64 read numa-node-id and distance-map when booted from a device tree.
  • Memory. Frames are pooled per node. A processor's frame cache refills from its own node and falls back in distance order; a free returns a frame to the node it came from, whichever processor frees it. Linear-memory slots, fiber stacks and kernel per-processor structures are backed from the node of the processor they serve.
  • Scheduling. Work stealing prefers a victim on the thief's node before a remote one, and an instance's home is the node its memory was committed on.
  • Observation. The boot line reports the node count, each node's memory and processors, and helios:system/stats reports per-node free memory and the count of off-node allocations.

Each new structure states which of its operations run on the owning processor and which arrive from any (§3.5).

Done when

  • Each architecture boots a two-node guest (-numa node with a memory backend per node) in its smoke lane and the boot line matches the topology QEMU published.
  • A kernel test drives allocation on a two-node memory plan and asserts node-local placement, distance-ordered fallback and that a cross-processor free returns the frame to its node.
  • A single-node boot shows no regression on the paired lane.
  • The performance claim is measured on a host with more than one physical node. An emulated topology on a single-node host has uniform memory latency, so it proves placement and nothing about speed.

Activity

  1. added
    enhancementNew feature or request
    perfruntime performance
    smpmulti-core correctness and scaling
    on Nov 11, 2026
  2. lexoliu commented on Nov 11, 2026

    @lexoliu
    OwnerAuthor

    NUMA topology for Helios (#516): design

    All file:line citations below are against origin/dev at 6716675b. A citation into a third-party crate gives the crate version and a path inside its published source.

    0. Summary

    hal gains a numa module with four parts: a NumaTopology trait that every backend implements, a dense NodeId, NodeDistance in SLIT units, and a SingleNode implementation. MemoryRegion becomes a struct that carries its NodeId. The firmware readers live in a new #![no_std] library crate, firmware/ (helios-firmware): ACPI SRAT/SLIT through the acpi crate for x86-64 and aarch64, and devicetree numa-node-id/distance-map through fdt for aarch64 and riscv64.

    The kernel copies the description once, at prime_bootstrap_allocator, into an immutable Locality. It then splits the user pool into one buddy pool per node and gives each processor's frame cache one list per node, following Linux's per-zone per_cpu_pages. Allocation follows a distance-ordered memory order per node. Per-processor structures are carved from a node-local arena. The executor's single global ready queue becomes one queue per node, drained in distance order. An instance's home node is fixed at its first processor turn, which is where Wasmtime commits its linear memory.

    A machine with no NUMA description is SingleNode: one pool, one queue per node, an empty remote order. No branch in the kernel distinguishes it.

    1. Consumer inventory

    1.1 How memory reaches the allocator today

    • Region type. hal::memory::MemoryRegion is NonNull<[u8]> (hal/src/memory.rs:3).
      • hal::memory::MemoryMap (hal/src/memory.rs:5-11) has no implementor and no consumer. git grep -w MemoryMap finds only its definition and an unrelated generic parameter name in hal/src/boot.rs:98.
    • The real entry point is helios_kernel::prime_bootstrap_allocator(regions, processor_count) (kernel/src/lib.rs:1275-1291). It is called by:
      • x86: x86/src/lib.rs:182
      • aarch64: aarch64/src/lib.rs:563
      • riscv: riscv/src/lib.rs:676
      • hosted: through init → bootstrap_init (kernel/src/lib.rs:1153, 1305).
    • Platform::memory_regions is dead on three backends.
      • aarch64 passes core::iter::empty() (aarch64/src/lib.rs:621).
      • x86 passes real regions (x86/src/lib.rs:271), but init takes the BOOT_INITIALIZING branch and never reads them (kernel/src/lib.rs:1155).
      • riscv passes them (riscv/src/lib.rs:729), and they are ignored the same way.
    • init_allocator (kernel/src/lib.rs:1203-1273):
      • copies the regions into a 128-entry stack ArrayVec (MAX_BOOT_MEMORY_REGIONS, kernel/src/lib.rs:811);
      • totals them into a machine-wide BootMemoryPlan;
      • gives the kernel heap its boot share from the front of the map (BootRegionSplitter, kernel/src/memory/policy.rs:246-283);
      • seeds one UserMemoryPool with the rest (kernel/src/lib.rs:1271);
      • sizes three per-processor tables from processor_count (kernel/src/lib.rs:1249-1257).
    • Duplicated backend code. x86 boot_memory_regions (x86/src/lib.rs:691-714) and aarch64 boot_memory_regions (aarch64/src/lib.rs:2189-2207) are the same function. §1 requires the change that notices this to fix it.

    1.2 Consumers and what each needs from the topology

    Consumer What it does today What it needs
    UserMemoryPool (kernel/src/memory/user.rs:20-32) One IrqSafeMutex<FramePool> buddy, one FrameSlabCache, one ReportedFrames bitmap, one total_bytes. Node of every boot region (to build per-node pools). Node of a frame on free and on reported-bit lookup. Node of the executing processor and its memory fallback order on allocation.
    FramePool::initialize (kernel/src/memory/frame_pool.rs:44-85) One SummaryBuddyAllocator over the span from the lowest to the highest region, with metadata inside that span. Called once per node with that node's regions only, so the metadata is node-local and no span covers another node's memory.
    allocate_raw (user.rs:192-232) Single frame: tries the caller's slab shard, then the buddy. On buddy failure it drains every shard and retries. Processor → node, then a memory order per node: local node first, others by distance. The drain is per node.
    try_allocate_frame_on (user.rs:250-273), fault path Try-locks only: the slab shard, then the buddy. The same memory order, try-lock at every step.
    FrameSlabCache (kernel/src/memory/frame_slab.rs:32-40, 118-221) One shard per processor plus a shared fallback shard. deallocate_on(processor) pushes into the freeing processor's shard (frame_slab.rs:155-162), whatever the frame's origin. Per-shard capacity is total/64/(shards+1) (frame_slab.rs:200-206). Node of the freed frame, so a foreign free stays attributed to its node. One list per (processor, node), sized from that node's total.
    deallocate_with_processor (user.rs:289-308) Single frame goes to the shard; anything else goes to the one buddy. Node of the address, to pick that node's buddy.
    lend_to_kernel_heap (user.rs:75-85) One-way power-of-two block from the pool to the kernel heap, called from inside the global allocator (kernel/src/lib.rs:587). Node of the executing processor, through the memory order. Off-node accounting.
    largest_servable_bytes (user.rs:167-186) Probes one buddy. Maximum over the nodes.
    BootMemoryPlan/BootRegionSplitter (policy.rs:150-293) Machine-wide plan; the kernel boot share comes from the front of the map. Regions ordered with the bootstrap processor's node first, so the boot heap is local to the processor that uses it. The plan itself stays machine-wide.
    frame_reserve (kernel/src/memory/frame_reserve.rs:23-49, 270-315) Per-processor lock-free stack of zeroed frames for the fiber-stack fault path. Refilled on its owner (kernel/src/lib.rs:899, frame_reserve.rs:287, 305). Nothing new: refills go through the per-node allocation path, so the reserve becomes node-local by construction. Its shard block must be node-local (per-processor structure).
    UserMemoryOwners (kernel/src/memory/owner.rs:57-77) Per-processor "who is committing" cells. Node-local block only.
    Fiber stacks (kernel/src/memory/fiber_stack.rs:201-243, 455-549) Cold-slot pages come from frame_reserve::take_frame(current_processor()) at fault time (fiber_stack.rs:471-472, 531). Warm slots keep the pages of whoever last ran on them (fiber_stack.rs:18-32). take_slot takes the lowest slot in a state (fiber_stack.rs:557). Per-processor demand_commits/announced arrays (fiber_stack.rs:228, 232). Fault frames: node-local through the reserve. Warm-slot choice: the node a slot's pages are on, so a claim prefers a warm slot on the claimant's node. Per-processor counters: node-local blocks.
    Linear-memory slots Wasmtime's pooling allocator reserves the slots (kernel/src/wasmtime_adapter/engine.rs:212-244). Physical frames arrive through AddressSpace::commit (hal/src/vmm.rs:332) on the processor running the store's turn, via allocate_user_frame_zeroed_on(current). Call sites: x86/src/vmm.rs:476-480; aarch64/src/vmm.rs:658; riscv/src/vmm.rs:580. Every backend ships the no-image hook (x86/src/vmm.rs:1329; aarch64/src/vmm.rs:1676; riscv/src/vmm.rs:1457), so data segments are copied into the instance's own pages. Nothing at the slot level. Commit already happens on the instance's processor, so per-node allocation makes it first-touch. Unverified: that the pinned Wasmtime revision decommits a slot on release. The default linear_memory_keep_resident: 0 was read from a different Wasmtime branch than docs/wasmtime.md pins. What placement does need is the instance to keep running on its home node (§6).
    Relocation (compaction mechanism) x86 build_relocation_plan allocates replacement frames with alloc_user_frame() on the current processor (x86/src/vmm.rs:495-520). Node of the old frame: a relocation that moves a page across nodes undoes placement.
    Balloon (kernel/src/memory/balloon.rs:261-551) Generic over Pool: PhysFrameAllocator. Inflation budget against a machine-wide floor (balloon.rs:354-358). Inflates with pool.allocate (balloon.rs:386). Reporting is one free_runs pass over the whole pool (balloon.rs:509-550). Per-node free counts and floors. Allocation on a chosen node. Per-node reporting. Deflate returns each run to its own node.
    ReportedFrames (kernel/src/memory/reported.rs:45-68) One bitmap over the pool's span. One per node pool. No topology query beyond that.
    KernelPhysFrameAllocator (kernel/src/memory/pmm.rs:38-216) A second PhysFrameAllocator over buddy_system_allocator. Production use is only a PhantomData (kernel/src/exec/compaction.rs:108); otherwise tests (hosted/src/pmm_tests.rs). Nothing. It is dead code duplicating UserMemoryPool's role (decision 9.4).
    Executor (kernel/src/exec/executor.rs:101-113, 860-992, 1268-1311, 1359-1391) Per-processor local_queues, local_ready_counts, idle, task_arenas, plus one global MPMC queue every processor drains, a global ready count and a global wake cursor. There is no queue-to-queue stealing today. The global queue is the balancing mechanism: a pushed task wakes one processor by cursor (executor.rs:1283-1310), and every processor pops it. Node of each processor. Per-node queues. A scheduling order per node by distance. Each task's home node.
    Task arena (executor.rs:171-219, 1369-1378) Per-processor buddy arena, "carved from the kernel heap at boot" (executor.rs:210-215). Node-local backing (per-processor structure).
    InstanceSpawner (executor.rs:1192-1225) and ProcessorTurns::begin (kernel/src/instance.rs:1103-1128) Instance tasks are global tasks. begin opens the owner scope under which the instance's commits happen. A per-instance home cell, set where the first commit happens, read by the instance's schedulers.
    Network shards (kernel/src/network/service/mod.rs:676-698) One shard per processor. Node-local block for the shard.
    Backend per-processor blocks x86 ProcessorSlot runtimes in one Box<[...]> and AP kernel stacks from allocate_aligned_zeroed (x86/src/smp.rs:347-434, 387). aarch64 slot array (aarch64/src/lib.rs:367-379). Node-local stacks and runtime blocks.
    Wasmtime table pool Committed whole at engine build (engine.rs:197-207), on whichever processor builds the engine. Out of #516's scope; see 9.8.

    1.3 Processor identity the description must map from

    • x86. ProcessorId is the MADT entry order with the first entry as bootstrap: x86/src/smp.rs:347-420, following acpi-6.1.1 src/platform/interrupt.rs:108-160 ("The first processor is the BSP"). The pre-heap count walks the same MADT (x86/src/lib.rs:604-634). The SRAT key is the APIC/x2APIC id.
    • aarch64. ProcessorId is the bootstrap first, then Limine's MP order (aarch64/src/lib.rs:343-379), keyed by MPIDR. ACPI SRAT GICC affinity is keyed by ACPI processor UID, which MADT GICC maps to MPIDR (walked at aarch64/src/platform/acpi.rs:209-236). The devicetree cpu reg is MPIDR.
    • riscv. ProcessorId is the hart id (riscv/src/lib.rs:642), and the cpu node reg is the hart id.

    ProcessorId is a dense index everywhere the kernel uses it: secondaries start for 0..configured_processors (kernel/src/lib.rs:1363-1367), and per-processor tables are indexed by id() (frame_slab.rs:213, executor.rs:803).

    2. Published prior art

    Only the points that name a crate source line were read from source. Points marked (unsure) are to be checked against the Linux tree before the code that depends on them is written.

    2.1 Linux memory

    • Nodes and zonelists. Linux keeps a pg_data_t per node with zones inside it. Each node's zonelist is built by build_zonelists, which orders nodes with find_next_best_node by node_distance. Equal-distance nodes are staggered by a load penalty, so different nodes fall back to different peers rather than all to the lowest id (the exact penalty formula: unsure). Only populated zones enter a zonelist, so a memoryless node's CPUs allocate from the nearest node with memory (local_memory_node). Required by: the per-node memory order, its tie rule, and memoryless-node handling (§5.2).
    • Per-cpu pagesets. struct per_cpu_pages exists per zone, per CPU. Order-0 allocation pops the local CPU's pcp list of the zone being tried and refills from that zone's buddy in pcp->batch units. free_unref_page frees onto the current CPU's pcp list of the page's own zone (page_zone(page)), and free_pcppages_bulk drains it back to that zone. A remotely freed page therefore stays its node's page. Required by: per-(processor, node) slab lists (§5.3).
    • First touch. Under MPOL_DEFAULT, a page is allocated on the node of the CPU that faults it. Required by: allocation keyed on the executing processor (§5.2).
    • Accounting. /sys/devices/system/node/nodeN/numastat exposes numa_hit, numa_miss, numa_foreign, local_node, other_node and interleave_hit. Required by: the off-node counter (§7.2).
    • Per-cpu areas. setup_per_cpu_areas → pcpu_embed_first_chunk groups CPUs by node, using cpu_to_node and the distance callback, and allocates each group's units from that node's memory (memblock_alloc_node or the arch pcpu_fc_alloc). Required by: PerProcessor<T> (§5.5).
    • Validation and defaults. Linux's slit_valid() rejects a SLIT whose diagonal is not LOCAL_DISTANCE (10) or whose off-diagonal entry is not greater than 10; Linux then ignores the SLIT, where Helios fails fast. Without a distance table, node_distance defaults to LOCAL_DISTANCE/REMOTE_DISTANCE (10/20) on both the ACPI and the OF paths. x86 hands CPUs with no SRAT entry out round-robin, and Linux gives up on NUMA when SRAT covers less than the e820 RAM (1 MiB tolerance). Helios fails fast on both instead (§3.1, §4.1).

    2.2 Linux scheduling

    • sched_init_numa builds one NUMA sched-domain level per distinct distance value above the SMT/MC/PKG levels.
    • Idle (newidle) balancing walks domains from the innermost outward, so a CPU pulls from its own LLC and node before a remote node.
    • NUMA levels carry SD_NUMA, a larger imbalance_pct, and an allowed imbalance (imb_numa_nr/allow_numa_imbalance) that leaves small imbalances in place to keep tasks near memory (the thresholds: unsure).
    • Automatic NUMA balancing (hinting faults, task_numa_fault, preferred node) migrates tasks and pages toward each other.

    Required by: steal order by distance (§6.2). Hinting-fault migration is not in #516's scope; the instance home is fixed (§6.3).

    2.3 ACPI, as data

    The ACPI fields below were checked against the struct layouts in acpi-6.1.1 src/sdt/srat.rs:112-228.

    • SRAT. Subtable type 0 is Processor Local APIC Affinity:
      • proximity domain split between a low byte and three high bytes;
      • APIC id;
      • flags bit 0 Enabled;
      • clock domain.
    • Type 1 is Memory Affinity:
      • 32-bit proximity domain;
      • 64-bit base and length, each split into low and high halves;
      • flags bit 0 Enabled, bit 1 Hot Pluggable, bit 2 NonVolatile.
    • Type 2 is x2APIC Affinity (domain, x2APIC id, flags bit 0 Enabled). Type 3 is GICC Affinity (domain, ACPI processor UID, flags bit 0 Enabled). Types 4 and 5 are GIC ITS and Generic Initiator.
    • An entry without Enabled is ignored.
    • SLIT. A u64 locality count N followed by N×N u8 entries, where entry i*N+j is the distance from locality i to j (acpi-6.1.1 src/sdt/slit.rs:8-11).
      • Locality numbers are proximity domain numbers.
      • The diagonal is 10. Values 0-9 are reserved. 255 means unreachable.
      • The matrix may be asymmetric.

    2.4 Devicetree

    From Documentation/devicetree/bindings/numa.txt (not re-read for this design):

    • numa-node-id (u32) appears on cpu nodes, on memory nodes and on devices; the value is the logical node id.
    • A /distance-map node with compatible = "numa-distance-map-v1" carries distance-matrix, a list of <node-a node-b distance> u32 triplets.
    • Local distance is 10, and every inter-node distance must be greater than 10.
    • The binding states that distances are equal in both directions, and Linux of_numa_parse_distance_map_v1 sets both. (Unsure whether the binding treats one direction as sufficient.)
    • A machine has one memory@… node per memory range, and QEMU emits one per NUMA node.

    3. The hal contract

    3.1 New module hal/src/numa.rs

    //! Memory locality as the platform describes it.
    //!
    //! Concurrency contract: everything here is an immutable value or a
    //! read-only view over firmware tables the backend opened on the
    //! bootstrap processor. The kernel reads a description exactly once, on
    //! the bootstrap processor inside `prime_bootstrap_allocator`, before any
    //! secondary starts, and keeps its own copy; nothing in hal is read after
    //! that, so no implementation needs a lock or an atomic.
    
    use core::ops::Range;
    use crate::cpu::ProcessorId;
    
    /// The most nodes a description may name. Linux's x86-64 default
    /// (`NODES_SHIFT` = 6). A firmware naming more is refused by name.
    pub const MAX_NODES: usize = 64;
    
    /// A node: dense, `0..NumaTopology::node_count()`. Firmware numbering
    /// (ACPI proximity domains, devicetree `numa-node-id`) is mapped to it in
    /// ascending order of the firmware number.
    #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
    pub struct NodeId(u8);
    
    impl NodeId {
        pub const FIRST: Self = Self(0);
        /// Panics when `index >= MAX_NODES`.
        pub const fn new(index: usize) -> Self;
        pub const fn index(self) -> usize;
    }
    
    /// Relative memory latency between two nodes in ACPI SLIT units:
    /// `LOCAL` is 10, a remote node is above it, 255 is unreachable.
    #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
    pub struct NodeDistance(u8);
    
    impl NodeDistance {
        pub const LOCAL: Self = Self(10);
        /// What a platform that publishes no distance table means by "remote":
        /// Linux's `REMOTE_DISTANCE`, the convention both firmware paths use.
        pub const UNIFORM_REMOTE: Self = Self(20);
        pub const fn new(raw: u8) -> Self;
        pub const fn get(self) -> u8;
    }
    
    /// One physical range and the node it belongs to.
    #[derive(Clone, Debug, PartialEq, Eq)]
    pub struct MemoryAffinity {
        pub physical: Range<u64>,
        pub node: NodeId,
    }
    
    /// The platform's locality description.
    ///
    /// Contract an implementation states and the kernel checks once, failing
    /// with the offending entry named:
    /// - `node_count()` is in `1..=MAX_NODES`;
    /// - `processor_node(p) < node_count()` for every `p < processor_count()`,
    ///   and `processor_count()` equals `Cpu::processor_count()`;
    /// - `distance(n, n) == LOCAL`, `distance(a, b) > LOCAL` for `a != b`;
    /// - `memory_affinity()` ranges are disjoint, and every usable byte the
    ///   backend hands the kernel lies in exactly one of them.
    pub trait NumaTopology {
        type MemoryAffinities<'a>: Iterator<Item = MemoryAffinity> + 'a
        where
            Self: 'a;
    
        fn node_count(&self) -> usize;
        fn processor_count(&self) -> usize;
        fn processor_node(&self, processor: ProcessorId) -> NodeId;
        fn distance(&self, from: NodeId, to: NodeId) -> NodeDistance;
        fn memory_affinity(&self) -> Self::MemoryAffinities<'_>;
    }
    
    /// A platform with no NUMA description: one node holding every processor
    /// and all of physical memory.
    #[derive(Clone, Copy, Debug)]
    pub struct SingleNode { processors: usize }
    
    impl SingleNode {
        /// Panics on zero processors.
        pub fn new(processors: usize) -> Self;
    }
    
    impl NumaTopology for SingleNode {
        // node_count 1; processor_node -> FIRST; distance -> LOCAL;
        // memory_affinity -> once(0..u64::MAX, FIRST)
    }

    Why each piece exists:

    • Trait, not owned value. processor_node is a trait method rather than a stored array because the description is needed before the heap exists (x86/src/lib.rs:181-190, aarch64/src/lib.rs:560-563, riscv/src/lib.rs:676), and hal has no processor cap. x86 has none at all; riscv caps at 64 (riscv/src/lib.rs:170); aarch64 at 64 (aarch64/src/platform/mod.rs:54). A large cloud instance, where NUMA matters, exceeds 64.
      • The storage is the firmware tables themselves (caller-owned). That keeps heap allocation out of the contract (§3.1 of AGENTS).
      • The kernel copies the answers into its own heap once.
    • memory_affinity is what lets one shared helper tag regions (3.3). Backends do not each intersect ranges.
    • SingleNode is the issue's "one node, no special case": every consumer runs the same code over a one-element answer.
    • NodeDistance in SLIT units comes from SLIT and the devicetree binding (§2.3, §2.4). UNIFORM_REMOTE is Linux's default for a missing table (§2.1). Here it is data, not a fallback: a platform that publishes no table says nothing about relative remoteness.
    • No cfg(target_arch), no consumer names. Node, distance and affinity are hardware properties, which satisfies §1 and the hal_layering test (kernel/tests/hal_layering.rs:4-46).

    3.2 hal/src/memory.rs: before and after

    Before (hal/src/memory.rs:1-11):

    pub type MemoryRegion = NonNull<[u8]>;
    pub trait MemoryMap: Send + Sync + 'static { type Regions<'a>: Iterator<Item = MemoryRegion> + 'a where Self: 'a; fn regions(&self) -> Self::Regions<'_>; }

    After:

    /// Usable memory the kernel may own, in the kernel's direct map, and the
    /// node it sits on.
    #[derive(Clone, Copy, Debug)]
    pub struct MemoryRegion { span: NonNull<[u8]>, node: NodeId }
    
    impl MemoryRegion {
        /// # Safety
        /// `span` is usable memory nothing else owns, and it outlives the kernel.
        pub const unsafe fn new(span: NonNull<[u8]>, node: NodeId) -> Self;
        pub fn start(&self) -> usize;
        pub fn end(&self) -> usize;
        pub fn len(&self) -> usize;
        pub fn node(&self) -> NodeId;
    }

    MemoryMap is deleted, because it has no implementor or consumer (§1.1). The unsafe constructor moves the ownership promise that init_allocator currently assumes (unsafe { region.as_mut() }, kernel/src/lib.rs:1212) to the backend that knows it holds.

    3.3 hal/src/boot.rs: one tagging helper

    The new helper sits next to usable_region_segments (hal/src/boot.rs:149-163):

    /// The usable pieces of `region`, reserved ranges removed and each piece
    /// split at node boundaries, as kernel-owned regions.
    ///
    /// Panics naming the physical range when a usable byte lies in no
    /// affinity range: memory the kernel cannot place is memory it does not
    /// understand. (Linux tolerates <1 MiB uncovered and otherwise discards
    /// NUMA altogether; neither is a correct answer here.)
    pub fn usable_node_regions<'n, Numa: NumaTopology>(
        region: BootMemoryRegion,
        reserved: &BootReservedRanges,
        physical_memory_offset: usize,
        numa: &'n Numa,
    ) -> impl Iterator<Item = MemoryRegion> + 'n;

    x86 and aarch64 both call it and delete their identical boot_memory_regions (x86/src/lib.rs:691-714, aarch64/src/lib.rs:2189-2207). riscv's collect_memory_regions (riscv/src/lib.rs:976-1004) calls it per memory-node segment. Required by: §1's duplication rule, and every backend needing node-tagged regions.

    3.4 hal/src/lib.rs: Platform and ProcessorTopology

    Before (hal/src/lib.rs:191-246): Platform<Console, Cpu, Regions, WatchdogImpl> carries memory_regions: Regions, and Platform::new(console, memory_regions, cpu).

    After: Platform<Console, Cpu, WatchdogImpl> with no regions, and Platform::new(console, cpu) / with_watchdog(console, cpu, watchdog). Memory enters the kernel through exactly one call:

    pub fn prime_bootstrap_allocator<Regions, Numa>(regions: Regions, numa: &Numa)
        -> &'static memory::UserMemoryPool
    where Regions: IntoIterator<Item = MemoryRegion>, Numa: NumaTopology;

    This replaces (regions, processor_count: usize) at kernel/src/lib.rs:1275-1291. init on the bootstrap processor then requires BOOT_INITIALIZING; the BOOT_UNINITIALIZED → bootstrap_init arm (kernel/src/lib.rs:1152-1154) is removed. hosted primes explicitly before init, as every other backend already does. The reason for the change: a dead parameter on three backends (§1.1) plus a second entry would be two routes for one fact.

    ProcessorTopology (hal/src/lib.rs:60-97) is unchanged. init adds one assertion next to the existing processor-count assertion (kernel/src/lib.rs:1137-1142): the installed Locality reports the same processor count.

    Cpu (hal/src/cpu.rs:245-389) is unchanged. The executing processor's node has to be answerable from drop paths, the allocator and fault context, exactly like current_processor() (hal/src/cpu.rs:201-227). The kernel answers it with locality.node_of(current_processor()), one load from an immutable table. A Cpu method would add a second route to that answer.

    3.5 hal/src/pmm.rs: PhysFrameAllocator

    Before (hal/src/pmm.rs:148-210): allocate(count, zero), free_runs(min, max, visit), deallocate(range), stats().

    After:

    pub trait PhysFrameAllocator: Send + Sync + 'static {
        /// Nodes the allocator pools memory for, as `NumaTopology` named them.
        fn node_count(&self) -> usize;
        /// `count` contiguous frames from `node` and from no other node.
        fn allocate(&self, node: NodeId, count: usize, zero_first_use: bool)
            -> Result<PhysFrameRange, FrameAllocError>;
        /// Free-page reporting over `node`'s free runs.
        fn free_runs<Visit, Visited>(&self, node: NodeId, min_frames: usize, max_runs: usize, visit: Visit)
            -> impl Future<Output = usize> + Send
        where Visit: FnMut(PhysFrameRange) -> Visited + Send, Visited: Future<Output = ()> + Send;
        /// Returns `range` to the node it came from, whichever processor calls.
        fn deallocate(&self, range: PhysFrameRange);
        fn stats(&self, node: NodeId) -> FrameAllocStats;
    }

    Required by: the balloon (its only consumer, kernel/src/memory/balloon.rs:261-551), which must pick nodes and floors itself (§5.6). The module's SMP contract text (pmm.rs:8-13) is amended to say that each node's pool is fed from its own free lists and that a free returns to the frame's node.

    3.6 Call sites that must follow

    Item Call sites
    MemoryRegion x86/src/lib.rs:52, 696, 1005; aarch64/src/lib.rs:23, 621, 1195, 2193; riscv/src/lib.rs:149, 980, 1025-1029; hosted/src/runtime.rs:11, 292, 452; kernel/src/lib.rs:246, 1106, 1117, 1208, 1280, 1303; hal/src/lib.rs:6, 197, 212, 222, 366
    Platform::new/with_watchdog x86/src/lib.rs:271, 1005; aarch64/src/lib.rs:621, 1195; riscv/src/lib.rs:727; hosted/src/runtime.rs:333; kernel/src/test_support.rs (≈127); hal/src/lib.rs tests
    prime_bootstrap_allocator x86/src/lib.rs:182; aarch64/src/lib.rs:563; riscv/src/lib.rs:676; hosted (new call before runtime.rs:343)
    PhysFrameAllocator kernel/src/memory/user.rs:318-397; balloon.rs (all PhysFrameAllocator:: uses, 355-947); hal/src/vmm.rs:558 (doc); KernelPhysFrameAllocator and hosted/src/pmm_tests.rs (deleted, decision 9.4)

    4. Backends and the shared firmware crate

    4.1 Where parsing lives today, and the new crate

    Where it lives today:

    • x86 opens AcpiTables four times: x86/src/lib.rs:617 (pre-heap MADT count), x86/src/smp.rs:333, x86/src/iommu.rs:177, x86/src/rtc.rs:121. It parses MADT (lib.rs:620-631, ioapic.rs:65-101), FADT, IVRS/DMAR and MCFG.
    • aarch64 has one PlatformTables built pre-heap (aarch64/src/platform/mod.rs:286-303), dispatching to dt.rs or acpi.rs. Its module contract: built once on the bootstrap processor, then read-only (mod.rs:19-21).
    • riscv reads the FDT directly (riscv/src/lib.rs:625-641) and in each device module.
    • hal already holds one shared devicetree reader, hal::entropy::device_tree_seed (hal/src/entropy.rs:10).

    New crate firmware/, package helios-firmware:

    • #![no_std]; dependencies: hal, acpi, fdt, all already in the lockfile.
    • It belongs in the "Device and protocol libraries" row of §1: a boundary with three owners, x86, aarch64 and riscv.
    • Its job is turning firmware tables into hal NUMA values. It allocates nothing, so it runs pre-heap.
    • AGENTS.md §1's table and §7's test-units list need the new crate added in the same change.
    /// Backend-supplied mapping from the kernel's dense processor index to the
    /// identity the firmware keys affinity by (APIC/x2APIC id, MPIDR, hart id).
    pub trait ProcessorIdentities {
        fn processor_count(&self) -> usize;
        fn hardware_id(&self, processor: ProcessorId) -> u64;
    }
    
    pub struct AcpiNuma<'t, H: acpi::Handler, P: ProcessorIdentities> { /* SRAT, SLIT, MADT views; domain list: ArrayVec<u32, MAX_NODES> */ }
    pub struct DeviceTreeNuma<'f, P: ProcessorIdentities> { /* &Fdt, node-id list: ArrayVec<u32, MAX_NODES> */ }
    impl NumaTopology for AcpiNuma<..> { .. }
    impl NumaTopology for DeviceTreeNuma<..> { .. }
    pub enum FirmwareNumaError { /* typed, thiserror: each names the entry it refused */ }

    The AcpiNuma reader:

    • No SRAT: node_count is 1 and the answers are SingleNode's. That is the same code path with one node, not a branch in the kernel.
    • Node ids: the Enabled entries' proximity domains, collected and sorted ascending. More than MAX_NODES distinct domains is refused.
    • Processors: LAPIC/x2APIC affinity keyed by APIC id (x86), and GICC affinity keyed by ACPI processor UID, which MADT GICC entries map to MPIDR (aarch64). A processor with no Enabled entry is refused; Linux's round-robin assignment is not copied (§2.1).
    • Memory: Memory Affinity entries with Enabled.
      • Hot Pluggable and NonVolatile entries contribute affinity only to the ranges the boot map lists as usable. Helios has no hot-add, and NVDIMM ranges are not usable RAM in the map, so the flags change nothing beyond their range.
    • SLIT: if present, indexed by proximity domain. The reader refuses a domain at or beyond N, a diagonal other than 10, or an off-diagonal value at or below 10 (§2.1 slit_valid, failing fast instead of ignoring).
      • No SLIT: UNIFORM_REMOTE.

    The DeviceTreeNuma reader:

    • Node ids: the numa-node-id values of cpu and memory nodes, sorted ascending.
    • Processors: the cpu node whose reg equals the hardware id.
    • Memory: every memory@… node's reg ranges with its numa-node-id.
    • Distances: /distance-map distance-matrix triplets, written in both directions and validated as above. A missing pair or a missing map is UNIFORM_REMOTE.
    • No numa-node-id anywhere: one node.

    Required by: #516 "Backends", and §1 (one SRAT/SLIT reader for x86 and aarch64, one devicetree reader for aarch64 and riscv).

    Blocker: acpi crate defects. The acpi crate (acpi-6.1.1, and still in 7.0.0-pre1, both read in the registry) has defects that make it unusable for this as published:

    1. MemoryAffinity::length() fills the high half from base_address_high (acpi-6.1.1 src/sdt/srat.rs:180-184). Any range above 4 GiB gets the wrong length.
    2. Slit::matrix_raw() checks size_of::<Slit>() + num_entries * num_entries with num_entries = N², so it checks N⁴ bytes (src/sdt/slit.rs:33-41). For N=2 it requires 60 bytes against a 48-byte table and returns an empty matrix for every multi-node SLIT.
    3. DistanceMatrix::distance() bounds-checks with <= and indexes i + j*N, transposed against its own doc comment (src/sdt/slit.rs:55-61).
    4. platform::numa::NumaInfo allocates Vecs (src/platform/numa.rs:14-19), so it cannot run pre-heap. The design does not use it; it uses Srat::entries() and Slit.

    Decision 9.1 covers how these are resolved.

    4.2 Per backend

    • x86 (x86/src/lib.rs:161-281).
      • Before prime_bootstrap_allocator, the backend builds AcpiNuma from the tables processor_count already opens (x86/src/lib.rs:604-634). Its ProcessorIdentities walks the MADT in entry order, which is the order AcpiPlatform assigns ProcessorIds in (§1.3).
      • It passes usable_node_regions(..) and &numa to prime.
      • The SRAT handler is the existing PhysicalOffsetAcpiHandler (x86/src/smp.rs:311-316), which maps by offset and allocates nothing.
    • aarch64 (aarch64/src/lib.rs:540-632). PlatformTables (mod.rs:286-332) gains numa(&self, ids) -> PlatformNuma, with two implementations of the trait, one per firmware path. The ids come from Limine's MP response in the existing bootstrap-first order (lib.rs:343-366), computed pre-heap from MP_REQUEST like aarch64_processor_count (lib.rs:475-485).
      • PlatformNuma is a struct generic over the two readers, not an enum, per §3.2.
      • The DT and ACPI paths stay exclusive, per mod.rs:12-17.
    • riscv (riscv/src/lib.rs:618-740). DeviceTreeNuma with identity hart ids. collect_memory_regions walks every memory node, which fixes the repair in 9.6.
    • hosted. SingleNode::new(machine.processor_count()), primed in the bootstrap thread before init.

    5. Kernel memory

    5.1 Locality (new, kernel/src/memory/locality.rs)

    pub struct Locality {
        processor_nodes: Box<[NodeId]>,            // index: ProcessorId
        node_processors: Box<[Box<[ProcessorId]>]>,// index: NodeId
        distances: Box<[NodeDistance]>,            // node_count², row = from
        scheduling_order: Box<[Box<[NodeId]>]>,    // per node: every node, self first, then by distance
        memory_order: Box<[Box<[NodeId]>]>,        // per node: nodes with memory, by distance
        frame_nodes: FrameNodes,                   // physical/direct-map address -> NodeId
    }
    • Construction. Built inside init_allocator, right after the first kernel-heap region is added. The heap is needed for the boxes, and the per-processor tables configured there need it (kernel/src/lib.rs:1247-1258).
    • Validation. Every NumaTopology contract clause in 3.1 is checked here, with a panic that names the processor, node pair or range at fault.
    • Order ties. Equal distances are ordered by (n − self − 1) mod node_count. That spreads equal-distance fallback the way Linux's find_next_best_node staggering does (§2.1), without load state.
    • FrameNodes. One sorted ArrayVec<(usize, NodeId), MAX_BOOT_MEMORY_REGIONS> holding the start of each run of same-node regions, merged across regions. Lookup is partition_point. A single-node machine has one entry, so the lookup is trivially constant time.
    • Publication. Locality lives inside the install-once UserMemoryPool (published Release/Acquire through USER_MEMORY_POOL, user.rs:405, 447-475). That follows the existing install pattern rather than adding a static.
    • Concurrency. Built on the bootstrap processor before any secondary starts. Immutable after that. Read without synchronization from any processor and in fault and interrupt context.

    5.2 Per-node pools

    pub struct UserMemoryPool {
        nodes: Box<[CachePadded<NodePool>]>,    // index: NodeId
        frame_cache: FrameSlabCache,            // per processor, per node lists (5.3)
        off_node: PerProcessor<AtomicU64>,      // 7.2
        locality: Locality,
    }
    struct NodePool {
        heap: IrqSafeMutex<FramePool>,          // this node's regions only
        reported: ReportedFrames,               // bitmap over this node's span
        total_bytes: AtomicUsize,
    }

    Initialization. init_allocator groups the splitter's user shares by MemoryRegion::node() and calls FramePool::initialize once per node that has memory (frame_pool.rs:44-85, unchanged). The buddy metadata therefore lands inside that node's own regions. The splitter receives the regions ordered bootstrap-node-first, so the kernel boot share is local to the processor that uses it (policy.rs:249-252 keeps taking it "from the front").

    Allocation, for a processor P on node N:

    1. For each node M in memory_order[N]:
      1. pop P's list for M (5.3);
      2. take M's buddy;
      3. if the buddy fails, drain every processor's M-list into M's buddy and retry M.
    2. Only then move to the next M.

    Notes on this path:

    • Off-node counting. An allocation served by M ≠ N increments P's off-node counter.
    • Fault path. try_allocate_frame_on runs the same loop with try-locks only (user.rs:250-273).
    • No processor argument. Every processor: None path is resolved with current_processor(), which hal publishes for exactly this use, the global allocator included (AGENTS §3.5; hal/src/cpu.rs:201-227). That deletes the shared fallback shard (frame_slab.rs:38).
    • Requirements. The local-first rule is Linux's first touch (§2.1). The order and the memoryless-node handling are Linux's zonelists (§2.1). The per-node drain keeps today's rule, "drain before failing" (user.rs:208-215), without stripping other nodes' caches.

    Kernel-heap lending. lend_to_kernel_heap uses the executing processor's memory order. The kernel heap itself stays machine-wide (9.7).

    Free. deallocate_bytes_on and deallocate look up frame_nodes. A single frame goes onto the freeing processor's list for that node (5.3); anything larger goes to that node's buddy. ReportedFrames lookups go to the node's bitmap.

    Stats. stats() sums the nodes. stats_on(node) serves the per-node consumers (balloon, helios:system/stats).

    Largest block. largest_servable_bytes returns the maximum over nodes, probing in the caller's memory order.

    Relocation. A relocation replacement frame comes from the old frame's node and from no other. The new kernel API is allocate_user_frame_on_node(node); x86 build_relocation_plan (x86/src/vmm.rs:516) and the aarch64 and riscv relocate paths (aarch64/src/vmm.rs:1407, riscv/src/vmm.rs:1154) call it with node_of(old_phys). A node with no free frame refuses the relocation rather than migrating the page.

    Out-of-memory, pressure, compaction and swap triggers stay machine-wide. Allocation falls through every node before failing, so machine-wide exhaustion is the true out-of-memory condition. That matches Linux, where OOM is reached only when the whole zonelist is exhausted.

    5.3 Frame cache (FrameSlabCache), reshaped

    Before, there was one list per processor shard plus a fallback shard (frame_slab.rs:32-40). After:

    struct FrameSlabShard {               // one per processor, node-local block (5.5)
        lists: IrqSafeMutex<Box<[FreeList]>>, // index: NodeId; head ptr + count each
    }
    struct FreeList { head: *mut FreeFrame, cached: usize }
    • Ownership. P's shard holds P's cached frames for every node. A pop for node M takes lists[M]. A free of a frame on node M pushes lists[M] of the freeing processor's shard. This is Linux's per-zone per-cpu pageset (§2.1): a foreign free never touches the remote node's lock and never re-labels the frame.
    • Capacity. Each list holds at most node_total_frames(M).div_ceil(64).div_ceil(processor_count). That is the existing 1/64 retain rule (frame_slab.rs:26, 200-206) per node; the divisor loses its +1 because the fallback shard is gone. On one node, per-processor capacity grows by a factor of n/(n−1) (4/3 at four processors), the only single-node difference in this commit.
    • Concurrency.
      • Operations on the owner: pop, push, and try_allocate (fault path, try-lock).
      • From any processor: drain(node), which takes each shard's lock briefly, and cached_bytes(node), which reads the counts. The lock is IrqSafeMutex, as today (frame_slab.rs:8-10), because interrupt handlers allocate (§3.5 of AGENTS).
      • Cache lines: each shard is CachePadded. Its lock and lists are written by its owner, and by a drainer under the lock.

    5.4 Fiber stacks and the fault reserve

    • frame_reserve. No structural change. Refills (frame_reserve.rs:287, 305) now go through the node order of the owner, so the frames a fiber-stack fault maps are local to the faulting processor. The shard array moves to PerProcessor (5.5). Its contract is unchanged: every word written by the owner, the shard on its own line (frame_reserve.rs:23-34).
    • Fiber-stack arena.
      • FiberStackSlot (fiber_stack.rs:201-211) gains node: AtomicU8, written by a cold claim with the claimant's node. The claimant commits the top page (fiber_stack.rs:530-544) and faults most of the rest.
      • take_slot(SLOT_WARM) first scans for a warm slot on the claimant's node, then any warm slot, then a cold slot. Lowest-index-first is kept within each class (fiber_stack.rs:551-557).
      • demand_commits and announced (fiber_stack.rs:228, 232) become PerProcessor.
      • Contract addition: node is written by the claiming processor (any processor) on a cold claim and read by any claimant's scan. It sits in the slot's existing padded block, which claim and release already write.
      • Required by: kernel: no NUMA topology, so memory and tasks are placed without regard to node #516 "fiber stacks … backed from the node of the processor they serve", given that warm slots outlive the processor that faulted them (fiber_stack.rs:18-32).

    5.5 Per-processor structures: PerProcessor<T> (new, kernel/src/memory/per_processor.rs)

    /// One `T` per processor, each on its own cache line, each backed by
    /// memory of the node its processor is on.
    pub struct PerProcessor<T> { blocks: Box<[NonNull<CachePadded<T>>]> } // index: ProcessorId
    impl<T> PerProcessor<T> {
        pub fn new(locality: &Locality, init: impl FnMut(ProcessorId) -> T) -> Self;
        pub fn get(&self, processor: ProcessorId) -> &T;
        pub fn iter(&self) -> impl Iterator<Item = (ProcessorId, &T)>;
    }
    • Backing. The blocks are bump-carved from a per-node NodeLocalArena behind an IrqSafeMutex. The arena grows by one-way lends from its node's memory order, the same transfer lend_to_kernel_heap makes (user.rs:59-85), and is never freed. That matches the life of the structures it replaces: they are held in boot-time Once statics today (frame_slab.rs:39, executor.rs:55).
    • Requirement. This is Linux's per-cpu first chunk, grouped by node (§2.1).
    • Concurrency.
      • new may run on any processor, taking that node's arena lock; construction is rare.
      • get is lock-free from anywhere.
      • What each T contract says about its writers is that T's own business.

    Adopters:

    Structure Today
    frame slab shards frame_slab.rs:39
    frame reserve shards frame_reserve.rs
    owner cells owner.rs:59
    fiber per-processor counters fiber_stack.rs:228, 232
    executor per-processor block (local queue, ready count, idle, task arena header) executor.rs:102-109, 1370-1378
    network shards network/service/mod.rs:696
    x86 ProcessorRuntime and AP stacks x86/src/smp.rs:347-420, 387
    aarch64 slots aarch64/src/lib.rs:367-379

    Task arenas. The bytes (ArenaBytes, executor.rs:210-219) come from the owner's NodeLocalArena instead of the kernel heap.

    Accounting. The "Memory policy" boot line (kernel/src/lib.rs:1340-1348) gains per_processor_bytes. On a single node this commit therefore moves task_arena_bytes × processors from kernel_heap_bytes to per_processor_bytes with the machine total unchanged. The paired lane is the check (§8).

    The structures' own later allocations (sockets inside a network shard, for example) still come from the machine-wide kernel heap (9.7).

    5.6 Balloon and free-page reporting

    • Inflate. Each run is taken from the node with the largest free − total/PRESSURE_FLOOR_DIVISOR (balloon.rs:354-358, made per node) with allocate(node, ..). The guest keeps the same floor on every node, not just in sum. Taking from one node would push that node's instances off-node while the machine-wide number still looked healthy.
    • Deflate. deallocate(run) returns each run to its own node (balloon.rs:438).
    • Reporting. One pass per node, each bounded by that node's own spare (balloon.rs:509-524, per node) and walking that node's bitmap. This is Linux's per-zone page_reporting_process.
    • Unchanged. The host-facing statistics stay machine-wide, because the virtio-balloon statistics carry no node.
    • Concurrency. Unchanged (balloon.rs:48-53). The per-node choice is made inside the single service task.

    5.7 Contract summary for new and changed structures

    Structure On the owning processor From any processor Own cache line
    Locality none all reads; built once on the bootstrap processor before secondaries none needed; immutable
    NodePool none buddy take/return, drain target, lend, balloon, reporting (IrqSafeMutex); total_bytes yes, per node
    FrameSlabShard pop, push (any node list), fault try-pop drain(node), count reads yes, per processor
    Off-node counter increment (Relaxed) sum for stats yes, per processor
    NodeLocalArena none carve (IrqSafeMutex), boot-time only yes, per node
    PerProcessor<T> table none get (immutable pointer table) the blocks are
    FiberStackSlot::node none written on cold claim, read by every claim scan inside the slot's padded block
    NodeQueue (§6) none push by any scheduler, pop by own-node processors and thieves yes, queue and each counter
    InstanceHome (§6) none one CAS at first turn, read on every schedule no; written once

    6. Scheduling

    6.1 Queues

    ExecutorGroup (executor.rs:101-113), after:

    struct ExecutorGroup {
        processors: PerProcessor<ProcessorQueues>, // local queue, local ready count, idle, task arena
        nodes: Box<[CachePadded<NodeQueue>]>,      // replaces global_queue / global_ready_count / global_wake_cursor
        locality: &'static Locality,
    }
    struct NodeQueue {
        queue: ConcurrentQueue<Runnable>,          // bounded as today (READY_QUEUE_CAPACITY)
        ready: CachePadded<AtomicUsize>,
        wake_cursor: CachePadded<AtomicUsize>,
    }

    A one-node machine has exactly today's single global queue.

    6.2 Pop, steal and wake

    Run loop (executor.rs:860-906):

    1. Local first when its count is non-zero, otherwise the processor's own node first; then the other one of the two.
    2. Then each remote node in scheduling_order[N][1..], by distance.

    Remote stealing happens as soon as the thief's local queue and own node are empty. That is #516's rule: prefer a victim on the thief's node, then remote by distance. It also matches Linux's innermost-domain-first idle balancing (§2.2).

    Park test. has_ready_work (executor.rs:989-992) tests the local count and every node's count with SeqCst. The handshake in idle's module docs is unchanged: a processor publishes Parked, then re-tests everything it would drain.

    Wake on push to node M (replaces executor.rs:1283-1310):

    1. The cursor walks node_processors[M], skipping the current processor.
    2. The first candidate decides, exactly as today: IPI only if it is Parked.
    3. If node M has no candidate (its only processor is the pusher), the walk continues over the next node in scheduling_order[M].

    6.3 Home node

    Generic home. GlobalScheduler gains a home: GlobalScheduler<CpuImpl, Home: TaskHome>, with trait TaskHome { fn node(&self) -> NodeId; }. Variation through generics rather than an enum (§3.2). There are two implementations:

    • FixedHome(NodeId): Spawner::spawn (executor.rs:1139-1146) uses the spawning processor's node. A kernel task stays near where it was created instead of drifting with its wakers.
    • NoWeakArc<InstanceHome>: InstanceSpawner (executor.rs:1192-1225) carries the instance's cell. The launch path that builds the child's spawner (executor.rs:1206-1216) creates the cell and hands the same Arc to the registry entry (InstanceEntry, kernel/src/instance.rs:275-317).

    InstanceHome is an AtomicU8: a node id plus a provisional bit.

    • At launch it holds the launching processor's node, marked provisional.
    • ProcessorTurns::begin (instance.rs:1103-1128) is the point where the owner scope opens and Wasmtime commits the instance's linear memory, during instantiation, through AddressSpace::commit on this processor (owner.rs:1-15). On the instance's first turn, begin clears the provisional bit with a CAS and stores node_of(current_processor()).
    • From then on the home is fixed.

    Required by: #516 "an instance's home is the node its memory was committed on".

    Fibers that resume on another processor. A store's turn is one global task; its fiber resumes on whichever processor polls the task. Every wake pushes to the home node's queue, so the task resumes on the home node unless that node is saturated and a remote processor steals it.

    7. Observation and tests

    7.1 Boot lines

    These follow the existing "Kernel topology" line (kernel/src/lib.rs:1324-1328):

    NUMA topology nodes=2 distances=10,20;20,10
    NUMA node node=0 memory_bytes=1072693248 processors=0-1
    NUMA node node=1 memory_bytes=1073741824 processors=2-3
    
    • memory_bytes is the usable memory the node's pool was seeded with.
    • processors is a cpulist (0-1,4) formatted by a Display type, through tracing fields.
    • A single-node boot prints nodes=1 distances=10 and one node line.

    7.2 helios:system/stats

    In wit/deps/helios-system/stats.wit (record memory at lines 41-47):

      /// One node's share of the user pool.
      record memory-node {
        /// The node, as the boot line numbers it.
        id: u32,
        total-bytes: u64,
        /// Bytes this node's pool can still hand out, its processors'
        /// cached frames included.
        available-bytes: u64,
        /// Allocations requested by this node's processors and served from
        /// another node, since boot.
        off-node-allocations: u64,
      }
    
      record memory {
        total-bytes: u64,
        available-bytes: u64,
        pressure: memory-pressure,
        nodes: list<memory-node>,
      }
    • The counter is kept on the requesting side, like Linux's numa_foreign/other_node split. Its per-processor words are summed by node at sample time.
    • Bindings to update: kernel/src/wasmtime_adapter/component_host/mod.rs:4340 and :4376, the debugger and program WIT copies, and inspector/src/stats_tui.rs.

    7.3 Host unit tests

    All of these drive a synthetic plan over leaked host memory, using the existing pool_at construction (kernel/src/memory/user.rs:642-665) and test_processor_identity::set (kernel/src/lib.rs:1546-1562).

    1. Two-node plan, two processors (P0 on node 0, P1 on node 1).
      • Node-local placement: frames allocated on P0 lie in node 0's range, and P1's in node 1's.
      • Fallback: exhaust node 0, then a P0 allocation is served by node 1 and P0's off-node counter is 1.
      • Cross-processor free: allocate on P0, free on P1. Node 0's available bytes return to their prior value, node 1's are unchanged, and the frame sits on P1's node-0 list.
      • Drain before falling back: with node 0's buddy exhausted, the next P0 allocation is that same frame, drained from P1's node-0 list before any node-1 frame.
    2. Three nodes with an asymmetric matrix (row 0: 10, 30, 20). Exhausting node 0 serves from node 2 before node 1.
    3. Locality construction. Tie rotation; memory order skipping a memoryless node; each refused description (diagonal ≠ 10, off-diagonal ≤ 10, processor beyond node_count, uncovered usable range) panicking with its message.
    4. usable_node_regions. One segment that crosses a node boundary splits in two. An uncovered byte panics with its range.
    5. Executor.
      • With two nodes and processor 0 on node 0, tasks queued on node 1 and on node 0 run node 0's first, then node 1's.
      • On three nodes, steal order follows distance.
      • A push to node 1 whose only processor is the pusher wakes the nearest node's parked processor (the existing idle-state test harness, executor.rs:2007).
    6. firmware/. SRAT/SLIT and DTB bytes are built inside the tests from the spec layouts, so the test data is generated in the test rather than committed as binary fixtures. The tests cover:
      • sparse proximity domains;
      • a disabled entry;
      • a memory range above 4 GiB, which exercises the length_high defect;
      • a two-locality SLIT;
      • a devicetree distance-matrix given in one direction.

    7.4 QEMU arguments and the inspector knob

    vm gains --numa-nodes <N> and --numa-distance <D> (and the config-file fields numa_nodes, numa_distance), next to smp and memory (inspector/src/vm.rs:1373-1375, emitted at 4980-4981). AGENTS §5 says a missing knob is added to the inspector rather than passed through --qemu-arg (vm.rs:1403, 5028).

    • N must divide both --smp and --memory; anything else is refused by name.
    • --numa-distance emits -numa dist for every ordered pair, which makes QEMU publish a SLIT or /distance-map.

    For --memory 2G --smp 4 --numa-nodes 2 --numa-distance 20, on x86-64 q35, aarch64 virt (DT and --acpi) and riscv64 virt alike:

    -m 2G
    -smp 4,sockets=2,cores=2,threads=1
    -object memory-backend-ram,id=helios-node0,size=1G
    -object memory-backend-ram,id=helios-node1,size=1G
    -numa node,nodeid=0,memdev=helios-node0
    -numa node,nodeid=1,memdev=helios-node1
    -numa cpu,node-id=0,socket-id=0
    -numa cpu,node-id=1,socket-id=1
    -numa dist,src=0,dst=1,val=20
    -numa dist,src=1,dst=0,val=20
    

    Not verified against the lanes' QEMU builds, which stays open until the knob runs there:

    • that x86 q35 publishes SRAT with node 0 split around the 640 KiB-1 MiB hole;
    • that riscv virt derives its socket count from the NUMA node count and emits one PLIC and one memory node per socket;
    • that QEMU omits SLIT/distance-map without -numa dist.

    Smoke-lane assertion. Node count, processor lists and distances must match exactly. Each node's memory_bytes must fall in [size − 64 MiB, size], which absorbs firmware and kernel-image reservations.

    8. Staging: one PR, commits in order

    Single-node behaviour is unchanged through commit 7. Commit 8 changes only memory accounting on one node (5.5). The paired lane (bench-suite.yml, AGENTS §3.6) is the done-when check at the head.

    # Commit Files
    0a fix(riscv): read every devicetree memory node (own issue; 9.6) riscv/src/lib.rs (collect_memory_regions)
    0b fix(riscv): pick the PLIC whose contexts name the bootstrap hart (own issue; 9.6) riscv/src/net/mod.rs:372-381
    1 Remove KernelPhysFrameAllocator and its tests (9.4) kernel/src/memory/pmm.rs, kernel/src/memory/mod.rs:59, kernel/src/lib.rs:157, kernel/src/exec/compaction.rs:31,108, hosted/src/pmm_tests.rs
    2 hal contract: numa module, SingleNode, MemoryRegion struct, delete MemoryMap, usable_node_regions, Platform without regions, prime_bootstrap_allocator(regions, &numa). Every backend passes SingleNode and node-0 regions; x86/aarch64 boot_memory_regions folded into the hal helper. Kernel builds Locality; boot lines (7.1). hal/src/{numa,memory,boot,lib}.rs; kernel/src/lib.rs; kernel/src/memory/locality.rs; every backend's lib.rs; hosted/src/runtime.rs
    3 firmware/ crate with host tests; not yet wired firmware/**, root Cargo.toml members, justfile test-units
    4 PhysFrameAllocator per node + per-node pools + per-(processor, node) slab lists + distance-ordered allocation + off-node counters + node-preserving relocation + balloon/reporting per node + tests 7.3.1-4 hal/src/pmm.rs, kernel/src/memory/{user,frame_pool,frame_slab,balloon,reported,policy}.rs, kernel/src/lib.rs (init_allocator), {x86,aarch64,riscv}/src/vmm.rs (relocation only)
    5 Executor per-node queues, TaskHome, InstanceHome, steal order and wake routing + tests 7.3.5 kernel/src/exec/executor.rs, kernel/src/instance.rs, kernel/src/component/runtime.rs, the launch-path spawner sites
    6 Wire firmware readers into backends (multi-node descriptions now reach the kernel) x86/src/lib.rs, aarch64/src/{lib.rs,platform/*}, riscv/src/lib.rs
    7 helios:system/stats per-node record and bindings, inspector stats view wit/deps/helios-system/stats.wit, kernel/src/wasmtime_adapter/component_host/mod.rs, inspector/src/stats_tui.rs, guest WIT copies
    8 PerProcessor<T> + NodeLocalArena; adopters; task arenas; fiber warm-slot node preference kernel/src/memory/{per_processor,frame_reserve,owner,fiber_stack}.rs, kernel/src/exec/executor.rs, kernel/src/network/service/mod.rs, x86/src/smp.rs, aarch64/src/lib.rs
    9 Inspector --numa-nodes/--numa-distance; smoke-lane two-node boots (9.5) inspector/src/vm.rs, .github/workflows/ci.yml

    Independent implementers, without file overlap, once commit 2 has landed:

    • commit 3 (firmware/);
    • commit 9's inspector half (inspector/src/vm.rs only; it can start before commit 2);
    • commits 0a/0b (riscv only; they must land before commit 2 touches riscv/src/lib.rs);
    • commit 4 (kernel/src/memory, hal/src/pmm.rs);
    • commit 5 (kernel/src/exec, instance.rs).

    Commits 4 and 5 share only the read-only Locality accessor added in commit 2.

    Sequencing that cannot be split:

    • Commit 6 needs 3 and 4.
    • Commit 7 needs 4.
    • Commit 8 crosses memory, exec and the backends, so it runs after 4 and 5 under one implementer.
    • Commit 9's CI half runs last.

    9. Risks and open decisions

    9.1 The acpi crate's SRAT/SLIT defects (4.1).

    • Recommendation: fix all three upstream in one rust-osdev/acpi PR (MemoryAffinity::length high half; matrix_raw bound of N² bytes; distance bound < and index i*N+j), then bump to the release that carries it.
    • Reason: the crate is maintained and already in the lockfile (a 7.0.0-pre1 is out). A local parser would duplicate a maintained crate. Reading raw fields around a broken accessor is a workaround. Upstream main still carries all three as of 2026-11-10, with no issue or PR open against them.
    • Effect on staging: commit 6's ACPI half is blocked on that release; the devicetree half is not.

    9.2 Frame cache shape.

    • Recommendation: per-(processor, node) lists (5.3) rather than a remote free straight into the remote node's buddy.
    • Reason: it is Linux's proven shape, and it keeps a migrating instance's teardown from taking the remote node's pool lock once per page. Each shard costs 16 bytes per node.

    9.3 Instance home before the first turn.

    9.4 KernelPhysFrameAllocator.

    • Recommendation: delete it in commit 1.
    • Reason: its only production reference is a PhantomData (compaction.rs:108). Keeping it would mean porting a second, unused frame allocator to the per-node contract (§3.1, no duplicated paths).

    9.5 CI lanes.

    9.6 Pre-existing riscv defects a two-node riscv boot hits.

    • Fdt::memory() returns the first node named memory only (fdt-0.1.5 src/lib.rs:238-239, find_node doc at 277-282). collect_memory_regions (riscv/src/lib.rs:641, 976-1004) would silently drop node 1's memory.
    • discover_plic_context takes the first PLIC node (riscv/src/net/mod.rs:376). On a two-socket virt that may be the other socket's PLIC, which has no context for the bootstrap hart (unverified which PLIC QEMU writes first). The backend then refuses the UART (riscv/src/lib.rs:846-853).
    • Both are repairs the PR cannot be proven without, so each gets its own issue and commit (§9 of AGENTS).

    9.7 Kernel heap stays machine-wide.

    9.8 Wasmtime table pool.

    • It is committed whole at engine build (engine.rs:197-207), so every instance's funcref tables sit on the build processor's node and are read on every call_indirect.
    • Fixing it needs a per-slot placement or recommit hook in the pooling allocator, which is a fork primitive. Per §1 it is reported, not designed here.
    • The primitive's shape: a PoolingAllocationConfig hook called on table-slot allocation with the slot's range, so the embedder can commit it on the current processor.

    9.9 Eager remote stealing.

    Not yet verified: the QEMU behaviours listed in 7.4 and 9.6, the Linux details marked (unsure) in §2, and the pinned Wasmtime revision's linear_memory_keep_resident default (1.2).

  3. lexoliu commented on Nov 12, 2026

    @lexoliu
    OwnerAuthor

    The aarch64 image of CI run 43160598414 (811651e) booted under HVF on an M1 host, device tree, 4 vCPUs and 2G, with and without --numa-nodes 2 --numa-distance 20. Both reach the guest shell.

    One node:

    NUMA topology nodes=1 distances=10
    NUMA node node=0 memory_bytes=1740943360 processors=0-3
    Per-processor memory per_processor_bytes=28917760
    

    Two nodes:

    NUMA topology nodes=2 distances=10,20;20,10
    NUMA node node=0 memory_bytes=901316608 processors=0-1
    NUMA node node=1 memory_bytes=840011776 processors=2-3
    Per-processor memory per_processor_bytes=28983296
    

    The device-tree path reads the layout QEMU publishes: two processors per node, the user pool split across both, the distance map as given. The ACPI description still reports one node.

  4. lexoliu commented on Nov 13, 2026

    @lexoliu
    OwnerAuthor

    #598 lands the first three items: the two-node boots in every smoke lane (riscv64 and aarch64 from the device tree, x86-64 and aarch64 from ACPI), the two-node allocation tests in kernel/src/memory/user.rs, and two single-node paired runs with every compared row unchanged (43337651395, 43367031679).

    What keeps this open is the last item. Every CI runner and every development machine is one physical node, so placement is proven and its speed is not: the measurement needs a host with more than one node.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperfruntime performancesmpmulti-core correctness and scaling

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions