Overview
BLite's WAL currently operates within a single process. All synchronization primitives are in-process (SemaphoreSlim, ReaderWriterLockSlim, Interlocked counters) and every database file is opened with FileShare.None, causing a second process to receive an IOException immediately on open.
This issue tracks the implementation of multi-process WAL access via a shared sidecar file (.wal-shm), enabling N readers + 1 writer across OS processes on the same host.
The full plan is documented in
oadmap/v5/MULTI_PROCESS_WAL.md.
Root Causes
| Component |
Problem |
| PageFile.cs |
FileShare.None → second process gets IOException |
| WriteAheadLog.cs |
FileShare.None → same |
| _walIndex |
ConcurrentDictionary<uint, byte[]> — committed pages invisible to other processes |
| _nextTransactionId |
Interlocked on a local long — two processes generate duplicate IDs |
| _commitLock |
SemaphoreSlim(1,1) — in-process only |
| _checkpointRunning |
CAS on local int — two processes can checkpoint simultaneously |
Proposed Solution: .wal-shm sidecar file
A new binary file .wal-shm (opened as a MemoryMappedFile) shared by all participating processes. Contains only coordination metadata — no page data. Always reconstructible from the WAL if deleted or corrupted.
SHM fields: NextTransactionId, WalEndOffset, CheckpointedOffset, WriterOwnerPid, reader slot array (up to 32 slots × 16 bytes), double-buffered WAL index hash table (pageId → byte offset).
Locking model (performance-safe)
Writer threads are not affected by the OS lock. They post a PendingCommit to the group-commit channel and await a TaskCompletionSource. The OS lock is acquired at most once per batch (up to 64 transactions) by the single background group-commit task, after the in-process _commitLock is already held:
`
- await _commitLock.WaitAsync() ← SemaphoreSlim, microseconds
- _shm.TryAcquireWriterLock(timeout) ← OS lock, cross-process only
- … write WAL records + FlushAsync() …
- _shm.ReleaseWriterLock()
- _commitLock.Release()
`
In single-process mode (EnableMultiProcessAccess = false, the default) steps 2 and 4 are entirely skipped.
Platform Support
| Platform |
Writer lock |
Notes |
| Windows |
Mutex(Local\BLite_w_{hash}) |
Auto-released on process death |
| Linux |
cntl F_OFD_SETLK |
Auto-released on process death |
| macOS |
cntl F_OFD_SETLK |
Auto-released on process death |
| Android |
cntl F_OFD_SETLK |
Same APK processes only |
| iOS |
cntl F_OFD_SETLK |
App Group container required |
| WASM |
❌ Not supported |
No filesystem / no MemoryMappedFile |
All components are AOT-compatible.
Multi-File Mode
In multi-file mode (.idx + per-collection .db files) a single WAL is still used — splitting WALs per file would break ACID (a single insert touches both the collection file and the index file in the same transaction). The only multi-file-specific addition is parallel checkpoint I/O: pages for different physical files can be flushed concurrently via Parallel.ForEach.
Implementation Phases
| Phase |
Description |
| 0 |
EnableMultiProcessAccess opt-in flag — no behaviour change |
| 1 |
FileShare.ReadWrite + MemoryMappedFile.CreateOrOpen (Windows) |
| 2 |
WalSharedMemory infrastructure (MMF, atomic fields, reader slots, WAL index hash table) |
| 3 |
Cross-process writer lock inside _commitLock; txnId from SHM |
| 4 |
Shared WAL index + multi-process read path with local LRU cache |
| 5 |
Reader slot registration around read transactions |
| 6 |
Multi-process checkpoint with GetMinReaderOffset() safe boundary + parallel flush |
| 7 |
Crash recovery: stale writer PID detection, SHM corruption rebuild |
New files
- src/BLite.Core/Transactions/WalSharedMemory.cs
- src/BLite.Core/Transactions/WalSharedMemoryLayout.cs
- src/BLite.Core/Transactions/ReaderSlot.cs
- src/BLite.Core/Transactions/WalFrameIndex.cs
Out of scope
- Multiple simultaneous writers (requires BTree/SlottedPage page-latch safety first)
- WASM / browser
- Network filesystems (NFS, SMB)
- SHM encryption (SHM contains only offsets, never page data)
Overview
BLite's WAL currently operates within a single process. All synchronization primitives are in-process (SemaphoreSlim, ReaderWriterLockSlim, Interlocked counters) and every database file is opened with FileShare.None, causing a second process to receive an IOException immediately on open.
This issue tracks the implementation of multi-process WAL access via a shared sidecar file (.wal-shm), enabling N readers + 1 writer across OS processes on the same host.
The full plan is documented in
oadmap/v5/MULTI_PROCESS_WAL.md.
Root Causes
Proposed Solution: .wal-shm sidecar file
A new binary file .wal-shm (opened as a MemoryMappedFile) shared by all participating processes. Contains only coordination metadata — no page data. Always reconstructible from the WAL if deleted or corrupted.
SHM fields: NextTransactionId, WalEndOffset, CheckpointedOffset, WriterOwnerPid, reader slot array (up to 32 slots × 16 bytes), double-buffered WAL index hash table (pageId → byte offset).
Locking model (performance-safe)
Writer threads are not affected by the OS lock. They post a PendingCommit to the group-commit channel and await a TaskCompletionSource. The OS lock is acquired at most once per batch (up to 64 transactions) by the single background group-commit task, after the in-process _commitLock is already held:
`
`
In single-process mode (EnableMultiProcessAccess = false, the default) steps 2 and 4 are entirely skipped.
Platform Support
All components are AOT-compatible.
Multi-File Mode
In multi-file mode (.idx + per-collection .db files) a single WAL is still used — splitting WALs per file would break ACID (a single insert touches both the collection file and the index file in the same transaction). The only multi-file-specific addition is parallel checkpoint I/O: pages for different physical files can be flushed concurrently via Parallel.ForEach.
Implementation Phases
New files
Out of scope