Skip to content

perf(solana): drop two RPC round-trips from the x402 payment critical path - #19

Merged
VickyXAI merged 5 commits into
mainfrom
perf/solana-payment-rpc-cache
Jul 28, 2026
Merged

perf(solana): drop two RPC round-trips from the x402 payment critical path#19
VickyXAI merged 5 commits into
mainfrom
perf/solana-payment-rpc-cache

Conversation

@VickyXAI

@VickyXAI VickyXAI commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Every Solana payment made two serial RPC calls before it could sign — measured at ~107ms each against sol.blockrun.ai, ~212ms of the ~286ms median gap between the 402 landing and the paid retry going out. Base's equivalent gap is 2ms.

Found while benchmarking Base vs Solana settle latency: the reproducible Solana penalty was never settlement, it was this client-side gap.

getMint was fetching a constant

SPL Token fixes decimals in InitializeMint and ships no instruction to change it, so a mint's decimals is immutable. This path only ever transfers USDC, and the module already exported SOLANA_USDC_DECIMALS = 6 — while the payment path paid 107ms per call to fetch that same 6 over the network.

The call is gone outright, and getMint is out of the dynamic import, so it no longer reaches the bundle either.

getLatestBlockhash cached for 10s

Valid ~150 slots (~60s). The default RPC already caches this method 30s server-side, so a value can be 30s old on arrival; 10s keeps the worst case ~40s and leaves ~20s of settlement margin.

Cached per RPC URL rather than in a single slot, because endpoints genuinely differ on what "latest" is. One slot would mean a client alternating between a primary and a fallback refetches on every single call. The duplicate guard is deliberately not scoped this way — see below.

The non-obvious part: reusing a blockhash needs a duplicate guard

Two payments sharing a blockhash with identical economics compile to a byte-identical message. ed25519 is deterministic, so both carry the same signature and Solana rejects the second as already-processed. Verified directly before writing the fix:

same amount, price=1 (the existing constant):
  sig A: 3svEyz4eLd9ZQAuEMfA2P7353bK2r4w4C7qTLGbkbCAW
  sig B: 3svEyz4eLd9ZQAuEMfA2P7353bK2r4w4C7qTLGbkbCAW
  IDENTICAL -> true

Two same-priced calls in a row is a completely ordinary agent pattern, so each cached blockhash tracks the transactions already signed against it.

On a collision the fee nonce is tried first, without touching the network: the priority fee is nudged until the message is distinct. One step is +1 microLamport/CU over 8000 CU = 0.008 lamports, and feePayer is the facilitator, so it isn't the user's cost either. The top of the range, 65 microLamports/CU, sits three orders of magnitude under the facilitator's 50,000 ceiling (validateComputeBudgetLimits in @x402/svm), so it cannot trip verification.

Ordering matters here. My first version refreshed the blockhash first, and repeated same-priced calls then refetched every single time and kept the entire RPC cost — which is exactly what a benchmark loop does. The live numbers caught it.

Caught in review, before merge

Three duplicate-transaction paths, each reproduced before it was fixed.

The exhausted-nonce path re-emitted payment #1 (0b7449b)

When all 65 fee-nonce values were spent on one blockhash, the fallback fetched a fresh blockhash and rebuilt at the default price. But a forced refresh legitimately returns the same blockhash — that is what the RPC's 30s server-side cache does — and the cache correctly resolves that to the same entry, duplicate-guard state and all. Rebuilding at the default price there reproduced payment #1 byte for byte and re-emitted it as a silent duplicate, and stayed stuck that way until the blockhash actually rotated.

Reproduced before fixing: 70 same-priced payments produced only 65 distinct transactions, with #66 onward duplicating #1.

The nonce search now re-runs against the refreshed entry instead of restarting at the default price, and a genuinely exhausted range throws a named error rather than handing back a transaction that cannot land.

Concurrent payments collided (1c0b3f1)

The blockhash lookup read the cache into a local before awaiting the RPC and still trusted it afterwards. A burst is all inside getLatestBlockhash at the same moment, so each call missed what its siblings had stored, built its own entry with an empty issued record, and produced the same bytes. Measured with a deterministic mock: 8 concurrent same-priced payments produced 2 to 3 distinct transactions. The lookup now re-reads after the wait.

Two endpoints on one blockhash produced identical bytes (1c0b3f1)

The record of what had already been signed was filed per RPC URL. Solana never sees which URL served a blockhash — a transaction's identity is its blockhash plus its economics — so a client with a fallback RPC could send a payment through the primary and the byte-identical payment through the fallback, and the second is rejected as already-processed.

This one predates the caching change: the original single-slot code also created an empty record on a URL mismatch. The per-URL test missed it by discarding the fallback transaction instead of asserting on it.

The record is now keyed by blockhash and shared across endpoints, while endpoints keep their own blockhash and TTL, which they do genuinely need. It is bounded to 8 recent blockhashes — one expires in ~60s, so an older entry cannot describe a transaction that could still land, and a long-running process no longer accumulates every blockhash and transaction it ever signed.

The tests could reach the live gateway

vi.doMock + vi.resetModules() per test rebuilds the module registry while createSolanaPaymentPayload's dynamic import is already in flight. Under Promise.all some imports resolved to the real @solana/web3.js and called the live gateway — visible as blockhashes the stub never returned, and about 30% flakiness. Both Solana test files now use a hoisted vi.mock, so they cannot reach the network.

Measured, live gateway, 24 real payments per run

Alternating chains, two models, before vs after:

before after
solana sign gap (median) 286ms 7ms
solana total, openai/chat-latest 1686ms 1412ms
solana total, anthropic/claude-haiku-4.5 1532ms 1143ms

Remaining outliers in the sign gap (~130ms, 2 of 12) are the 10s TTL expiring and refetching — as designed. 24/24 payments settled clean, no errors.

Verification

228 unit tests pass across 5 consecutive full-suite runs, no flakes; typecheck, lint and build clean. Ten tests cover the caching and the hazards, and each behaviour is mutation-checked — reverting it fails exactly its own test:

revert this and this test fails
the re-read after the RPC wait keeps eight concurrent same-priced payments distinct
blockhash keying of the issued record does not repeat bytes across two endpoints
the nonce re-search after refresh fails loudly once the fee-nonce range is spent
the TTL check refetches once the blockhash TTL expires
per-URL blockhash entries keeps a separate blockhash entry per RPC URL

Ships as 3.8.4 (VERSION, package.json, src/version.ts, CHANGELOG).

Follow-ups, deliberately not in this PR

The nonce search is O(n) per payment. The Nth same-priced payment on one blockhash re-signs from the default price upward, so the 65th does 64 wasted signs before falling back. Correct, but it is linear work on exactly the burst pattern this PR optimizes for. A per-entry nonce cursor would make it O(1), at the cost of giving up the "first payment on a blockhash always uses the default price" property.

Two separate processes on the same wallet can still collide. Same-amount payments from different processes can receive the same server-cached blockhash. The record is process-local, so that case is unchanged — neither worse nor better. Worth addressing if concurrent same-wallet workers are a real deployment shape.

No in-flight coalescing at the TTL boundary. N concurrent payments crossing the boundary each issue their own getLatestBlockhash. Still strictly better than before, where every payment fetched unconditionally.

1bcMax added 2 commits July 27, 2026 16:19
… path

Every Solana payment made two serial RPC calls before it could sign, measured
at ~107ms each against sol.blockrun.ai (~212ms of the ~286ms median gap between
the 402 landing and the paid retry going out). Base's equivalent gap is 2ms.

getMint was fetching a constant. SPL Token fixes `decimals` in InitializeMint
and ships no instruction to change it, so a mint's decimals is immutable — and
this module already exported SOLANA_USDC_DECIMALS = 6, which nothing used. USDC
now never touches the network for it; an unknown mint pays the lookup once.

getLatestBlockhash is now cached for 10s. A blockhash is valid ~150 slots
(~60s) and the default RPC already caches this method for 30s server-side, so
a value can be 30s old on arrival; 10s keeps the worst case at ~40s and leaves
~20s of settlement margin.

Reusing a blockhash is only safe with a duplicate guard, which is the
non-obvious part. Two payments sharing a blockhash with identical economics
compile to a byte-identical message, and ed25519 is deterministic, so both
carry the SAME signature and Solana rejects the second as already-processed.
Two same-priced calls in a row is a completely ordinary agent pattern. Each
cached blockhash therefore tracks the transactions already signed against it.

On a collision the fee nonce is tried FIRST and the network is not touched:
the priority fee is nudged until the message is distinct. A step is
+1 microLamport/CU over 8000 CU = 0.008 lamports, and `feePayer` is the
facilitator, so it is not the user's cost. Only if all 64 steps collide do we
fetch a fresh blockhash — the pre-change behaviour. Ordering matters: doing the
refresh first meant repeated same-priced calls refetched every time and kept
the full RPC cost, which is exactly what a benchmark loop does.

Measured end to end against the live gateway, 24 real payments per run,
alternating chains, two models:

  solana sign gap   286ms -> 7ms median  (outliers are the 10s TTL expiring)
  solana total, chat-latest        1686ms -> 1412ms
  solana total, claude-haiku-4.5   1532ms -> 1143ms

212 unit tests pass, typecheck, lint and build clean. The new tests were
mutation-checked: disabling the duplicate guard fails two of them.
…n the nonce range is spent

Review fixes on top of the caching change.

Correctness bug in the exhausted-nonce path. When all 64 fee-nonce steps
collided, the code forced a blockhash refresh and reset the price to the
default — but a forced refresh legitimately returns the SAME hash (30s
server-side cache), which resolves to the same entry and issued set. Rebuilding
at the default price there reproduced payment #1 byte for byte, which is
exactly the duplicate Solana rejects. That case now throws instead of emitting
a transaction that cannot land.

The cache is now keyed by RPC URL rather than held in one slot. A client
alternating between a primary and a fallback endpoint would otherwise evict the
entry on every call and discard `issued` with it — silently disabling the
duplicate guard precisely when two endpoints fronting one cluster can hand back
the same blockhash.

getMint is dropped outright rather than cached: this path only ever transfers
USDC, whose decimals this module already exports.

Also records that the nonce ceiling (65 microLamports/CU) sits three orders of
magnitude under the facilitator's 50_000 limit.

Three new tests cover the exhausted range, TTL expiry under fake timers, and
per-RPC-URL isolation. 215 unit tests pass, typecheck, lint and build clean.
Re-measured live against the built SDK: 24/24 payments settled, solana sign gap
still 7ms median.
@VickyXAI

Copy link
Copy Markdown
Contributor Author

Pushed 0b7449b — review fixes on top of the caching change.

Correctness bug in the exhausted-nonce path. When all 64 fee-nonce steps collided, the code forced a blockhash refresh and reset the price to the default. But a forced refresh legitimately returns the same hash (30s server-side cache), which resolves to the same entry and issued set — so rebuilding at the default price reproduced payment #1 byte for byte. Exactly the duplicate Solana rejects. That case now throws rather than emitting a transaction that cannot land.

Cache keyed per RPC URL. Held in a single slot, a client alternating between a primary and a fallback endpoint would evict the entry every call and discard issued with it — silently disabling the duplicate guard precisely when two endpoints fronting one cluster can return the same blockhash.

getMint dropped outright rather than cached: this path only ever transfers USDC, whose decimals the module already exports.

Three new tests: exhausted range, TTL expiry under fake timers, per-RPC-URL isolation.

Re-verified end to end — 215 unit tests, typecheck, lint, build clean, and re-measured live against the built SDK: 24/24 payments settled, solana sign gap still 7ms median.

1bcMax added 3 commits July 27, 2026 22:39
Two duplicate-transaction paths, both reproduced before fixing.

Concurrent payments. getBlockhashEntry read the cache into a local before
awaiting the RPC and still trusted it afterwards. A burst is all inside
getLatestBlockhash at the same moment, so each call missed what its siblings
had stored, built its own entry with an empty issued record, and emitted the
same bytes. Measured with a deterministic mock: 8 concurrent same-priced
payments produced 2 to 3 distinct transactions. The lookup now re-reads after
the await. This one was mine, introduced in 0b7449b.

Cross-endpoint. The issued record was filed per RPC URL, but Solana never sees
which URL served a blockhash: a transaction's identity is its blockhash plus
its economics. A client with a fallback RPC could send a payment through the
primary and the byte-identical payment through the fallback, and the second is
rejected as already-processed. The record is now keyed by blockhash and shared
across endpoints; endpoints keep their own blockhash and TTL, which they do
genuinely need. This predates 0b7449b - the original single-slot code also
created an empty record on URL mismatch - and the per-URL test missed it by
discarding the fallback transaction rather than asserting on it.

The record is bounded to 8 recent blockhashes. One expires in ~60s, so an older
entry cannot describe a transaction that could still land, and a long-running
process no longer accumulates every blockhash and transaction it ever signed.

Tests moved to a hoisted vi.mock. The previous vi.doMock + vi.resetModules
pattern rebuilds the module registry while createSolanaPaymentPayload's dynamic
import is already in flight, so under Promise.all some imports resolved to the
real @solana/web3.js and called the live gateway - visible as blockhashes the
stub never returned, and about 30% flakiness. Unit tests here can no longer
reach the network.

All four behaviours are mutation-checked: reverting the snapshot read, the
blockhash keying, the TTL check, or the per-URL cache each fails exactly its
own test. 228 tests pass across 5 consecutive full-suite runs, typecheck, lint
and build clean.
… real

Two gaps from the previous commit, both found by re-checking claims rather
than re-reading the code.

The endpoint cache was never bounded. Only the issued-transaction record was.
A caller that builds a fresh URL per request - a rotating token in a query
string - left one dead entry behind forever. Both maps now trim to a window of
8, oldest first, which is what makes it safe: the entry in use is the one just
inserted, so it is never the one dropped.

The eviction test was vacuous. It rotated the stub blockhash 20 times but the
10s TTL meant the client refetched once, so all 20 payments reused a single
cached blockhash and the pruning path never ran at all. It passed for the wrong
reason. It now uses a distinct URL per round to force a real fetch, asserts the
fetch count to prove the rotation happened, and overflows both windows twice
over before checking the current blockhash still refuses to repeat itself.

Mutation-checked: pruning the newest key instead of the oldest drops the entry
being paid against and fails this test. 229 tests pass, typecheck, lint and
build clean.
The 3.8.4 entry said the issued record was bounded and said nothing about the
endpoint cache, which was still unbounded when that line was written. Both are
bounded now; say so, and say that the oldest entry is the one dropped, since
that is what keeps the blockhash in use from being pruned.
@VickyXAI
VickyXAI merged commit 2c6b57a into main Jul 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant