Feat/retire aimdb ws protocol

[lxsaah]

Retire aimdb-ws-protocol — every transport speaks AimX

Deletes the WebSocket-only wire protocol and converges the entire workspace on
the AimX-v3 envelope (aimdb-core::session::aimx). One codec, one error
mapping, one client demux — the WebSocket stack and the browser bridge now ride
the same session engine as UDS/serial/TCP and aimdb-client.

Implements design 047 (docs/design/047-retire-ws-protocol-converge-on-aimx.md).

Why

The workspace shipped two wire protocols for one feature. Every capability —
snapshots, acks, writes, queries, subscribe — existed twice, with two codecs and
two error vocabularies, plus a hand-rolled ~835-line browser demux duplicating
run_client. Both protocols already rode the same session engine, so the fork
was pure duplication.

Convergence removes a 353-line protocol crate, a 507-line codec, two error
mappings, and the browser demux, against a small additive AimX extension
(wildcard subscribe + two result-shape changes) that every transport now
inherits (CLI watch 'temp.#', MCP wildcard reads, future serial/TCP browsers).

What changed

Core (additive AimX extension)

• Wildcard / multi-record live subscribe. A pattern containing #/*
  matches the (builder-frozen) record set once at subscribe time, merges the
  matched records' update streams under one subscription id, and emits one
  snapshot per matched record on open.
• topic_match module — MQTT-style matching over dot-separated keys, shared
  by the core wildcard path and the WS fan-out bus.
• SubUpdate item type — subscription streams now carry
  { topic: Option<Arc<str>>, data, skipped } so each event names the concrete
  record that fired and threads buffer-lag counts through for drop detection.
• Wire additions: optional topic on event, sub on snap, an explicit
  subscribed ack frame (WS only; UDS/serial/TCP stay ack-implicit).
• Query/list result shapes: record.query → { records: [{topic, payload, ts}], total } (canonical QueryRecord row); RecordMetadata gains
  schema_type and entity.
• Version-gated hello — protocol bumped 2.0 → 3.0; an incompatible or
  version-less peer is refused at the handshake (RpcError::VersionMismatch).
• Loss accounting — buffer lag folds into skipped; the server pump does
  seq += skipped + 1; the client demux recovers loss as a seq shortfall.

WebSocket connector

• Speaks AimX on the transport (one codec blob per WS text frame — no extra
  framing). Dispatch converged on record.query / record.list.
• The fan-out bus carries (topic, payload) as Arc-shared updates; the
  per-connection codec envelopes each event (O(1) fan-out preserved).
• record.query falls back to the persistence-registered QueryHandlerFn when
  no custom handler is plugged in; record.list rows are stamped with schema
  names from the connector's StreamableRegistry.
• Removed: WsCodec + id↔topic maps, split_multi_topic, the pre-serialized
  Data-frame broadcast, with_raw_payload, ClientConfig::topic_routed_subs.

Browser bridge (aimdb-wasm-adapter::ws_bridge)

• Rewritten on run_client + ClientHandle over a web_sys::WebSocket-backed
  Connection/Dialer (single-threaded wasm Send/Sync wrappers). Reply and
  subscription correlation now exists exactly once, in core.
• JS API preserved: write, query, listTopics, onStatusChange, offline
  queue, and reconnect are all delegated to the engine.

Deletions

• aimdb-ws-protocol crate, WsCodec, protocol.rs shims, dead aimdb-client
  re-exports.

Breaking changes

• Wire protocol 2.0 → 3.0 — pre-3.x clients are refused at hello. Browser
  clients on the old ws-protocol wire must ship the rebuilt bridge; they fail
  closed against a 3.0 server.
• Removed builder knobs: with_raw_payload, ClientConfig::topic_routed_subs.
• record.query result shape changed from {values:[{record,value,stored_at}], count} to {records:[{topic,payload,ts}],total}.

Lands as one branch in a release already flagged protocol-breaking, one commit
per work item (no stacked PRs).

Testing

make check and make all green. New/updated coverage: AimX codec roundtrip
suite (frames + splice byte-for-byte), topic_match tests, topic_leaf
dual-separator, version-compatibility matrix, query_handler_shape,
UDS handshake_version, rewritten WebSocket e2e.rs (subscribe ack, wildcard
fan-out, late-join), and fan-out encode benchmarks.

Known limitations (accepted)

• Auto-subscribe under engine demux — server-seeded subscriptions carry
  server-chosen sub ids (counted down from u64::MAX); a run_client-based
  consumer drops events for ids it never issued, so engine clients (including the
  new bridge) subscribe explicitly. No user-visible change (the pro hub already
  subscribes by exact name).
• Dynamic record membership — wildcard matches are resolved once at subscribe
  time against the builder-frozen record set; MQTT-style late registration is
  deferred until core grows runtime registration.

[thaodt]

Snapshot delivery bypasses the new loss accounting: try_send errors are ignored and snapshots carry no sequence or gap metadata.
I reproduced this with a wildcard subscription whose dispatch returned 300 snapshots. After a server-side RPC barrier and before draining the stream, only 256 updates were available, exactly SUBSCRIBE_CHANNEL_CAP with no error or skipped signal.

That breaks the "one snapshot per matched record" and later event sequence numbers cannot reveal which initial states were lost. Could snapshot setup apply backpressure or surface an explicit overflow/error/gap?
I think we should add a regression covering more than 256 matched records.

[lxsaah]

Fixed: snapshots now carry seq in the subscription's sequence space and route through the same deliver() gap accounting as events, so overflow surfaces as skipped.
Regression added: oversized_snapshot_burst_surfaces_as_a_gap (300 records, verified failing pre-fix).

[thaodt]

Sharing the snapshot/event sequence space fixes a dropped snapshot once some later update is delivered. However, the tail of a snapshot-only burst is still silent until that later update exists.

I reran the 300-record case but checked accounting before injecting event_tx.send(...): the stream delivered 256/300 snapshots, every delivered item had skipped == 0 and then it remained open. A static subscription may never produce the "first event" used by the regression, so the consumer still cannot distinguish a complete snapshot from a truncated one.

Could snapshot completion itself provide backpressure or an explicit completion/overflow signal, without relying on a future live event? I think this thread should remain open until the snapshot-only case is covered.

[lxsaah]

Fixed: the burst's final snapshot is now flagged and the client reserves a sink slot for it. So your 300-record case delivers 255 + 1 flagged snapshot_end with skipped == 44. It is auditable the moment the burst ends, no live event needed.

Still uncovered and documented rather than fixed: the flag rides the final snapshot's frame, so it's lost if that frame fails to encode or decodes as malformed and a pattern matching zero records emits no burst at all.

[thaodt]

This creates an unbounded, frame-sized queue between the browser callbacks and the client engine. A fast server or stalled WASM executor can therefore grow Rust-owned browser memory w/o limit before the bounded subscription channels are reached.

Because this queue also carries RPC replies, silently dropping individual frames would leave calls unresolved. Could this be bounded and treat overflow as a transport failure, closing the connection and letting reconnect policy recover with a burst regression?

[thaodt]

This one seems was not addressed yet. The browser frame funnel is still an UnboundedReceiver, created with mpsc::unbounded, and on_message still uses unbounded_send.

The disconnect-handshake fix in this file looks good, but it does not bound inbound frame memory or define overflow behavior. Could we keep this thread open and either bound the funnel with overflow treated as a transport failure or explicitly document why unbounded browser-owned delivery is accepted?

[lxsaah]

Agreed. I bounded to 1024 frames. An overflow now closes the stream instead of dropping frames, which the engine reads as a disconnect (so pending calls reject rather than hang) and the re-subscribe re-syncs from scratch — which I'd argue isn't really a regression, since after an overflow there's no way to tell what was lost.

[thaodt]

When this timeout wins, dropping call drops only its oneshot receiver. If the request was already sent, drive_connection has stored the corresponding sender in its pending map and that entry is not removed until a reply or disconnect arrives.

A peer that keeps the connection open but never replies therefore retains one pending entry per timed-out request. Could the engine prune canceled oneshot senders or support explicit call cancellation? A repeated-timeout regression would be useful here.

[lxsaah]

Fixed: the engine now prunes canceled oneshot senders before each insert

[thaodt]

This bounds the sequential-timeout case, but cancellation still does not wake the engine.

I tested a concurrent batch where 1k calls are all tracked while their receivers are live, then all receivers are dropped after the final insert. With keep-alive disabled and no later call, all 1k cancelled entries remain until disconnect because there is no subsequent path that invokes prune_canceled.

There is also a performance side effect track_call runs HashMap::retain before every live insert, making a burst of N outstanding calls O(N^2).
Could we cover concurrent timeouts with keepalive disabled and use an explicit cancellation/wakeup, an amortized reclaim strategy or a bounded in-flight table? I think the current sequential regression does not fully close the original lifetime issue.

[lxsaah]

Both fair. I replaced the sweep with explicit cancellation: ClientHandle::call now emits ClientCmd::CancelCall { id } when its future drops, so the engine removes the entry by id in O(1) with no keepalive or later call needed (retain is gone, so a burst of N is O(N)) and the 1k concurrent batch with keepalive disabled is now a unit test that asserts every entry is freed.