Wire-Network · joshglogau · Aug 11, 2026
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -56,7 +56,7 @@ pnpm workspaces (no nx/turbo/lerna). All packages under `packages/`:
 |---------|---------|
 | `cluster-tool` (`@wireio/cluster-tool`) | THE core library: orchestration engine (PhaseGroup → Phase → Step → Report), process managers, chain clients, config/bind resolution, Steps palette, flow substrate (`FlowCLI`/`FlowScenario`), CLI |
 | `cluster-tool-shared` (`@wireio/cluster-tool-shared`) | Zod schema-first persisted shapes (`ClusterConfig`, `BindConfig`, `ClusterState`, `SignatureProviderConfig`, `ExternalOutpostConfig`, `ExternalClusterConfig`, `ChainTokenAmount`) behind the generic `SchemaCodec` (validate-both-ends serialize/deserialize) |
-| `flow-*` (13 packages) | One scenario each — standalone executables built on `FlowCLI.create(<Name>Scenario).run()`; batch-operator lifecycle (slashing/termination), collateral, reserves, emissions soak, node-owner NFT, yield distribution, and the six swap variants |
+| `flow-*` (14 packages) | One scenario each — standalone executables built on `FlowCLI.create(<Name>Scenario).run()`; batch-operator lifecycle (slashing/termination), collateral, reserves, emissions soak, node-owner NFT, yield distribution, the six functional swap variants, and the manual-only 10-actor swap/epoch stress diagnostic |
 | `debugging-shared` / `debugging-server` / `debugging-client-shared` / `debugging-client-tool` / `debugging-client-tool-tui` | OPP debugging surface: shared types + storage paths, ingest server, RPC client, CLI, TUI |
 | `test-app-server` | Fixture app server used by debugging tests |
 

diff --git a/README.md b/README.md
@@ -127,6 +127,7 @@ pnpm workspace (no nx/turbo/lerna); everything lives under `packages/`.
 | `cluster-tool` | `@wireio/cluster-tool` | Core harness: process managers, chain clients, bootstrap, **`wire-cluster-tool` CLI** |
 | `flow-operator-collateral-deposit` | `@wireio/test-flow-operator-collateral-deposit` | Node-operator collateral deposit + withdraw remit |
 | `flow-swap-with-underwriting` | `@wireio/test-flow-swap-with-underwriting` | Bidirectional SWAP (ETH ↔ SOL) with underwriting |
+| `flow-swap-epoch-stress` | `@wireio/test-flow-swap-epoch-stress` | Manual 10-actor ETH → SOL swap/epoch stress diagnostic (21 producers + 21 batch operators) |
 | `flow-swap-non-native-tokens` | `@wireio/test-flow-swap-non-native-tokens` | SWAP of non-native tokens (USDC / USDT / LIQ) |
 | `flow-swap-variance-revert` | `@wireio/test-flow-swap-variance-revert` | Swap variance-tolerance revert |
 | `flow-batch-operator-termination` | `@wireio/test-flow-batch-operator-termination` | Batch-operator termination via delivery underperformance |
@@ -216,6 +217,13 @@ and every live run is paired with the heartbeat monitor (see
 ./scripts/run-flow.mjs swap --wire-build-path … --ethereum-path … --solana-path …
 ```
 
+The long-running `flow-swap-epoch-stress` scenario is manual-only and excluded
+from the default E2E suite. It launches 10 simultaneous Ethereum → Solana swaps
+against 21 producer accounts and 21 batch operators, then records payout,
+UWREQ, Solana memory-log, and post-load epoch-liveness evidence. Run it locally
+with the canonical runner and heartbeat monitor shown in the package
+[README](packages/flow-swap-epoch-stress/README.md).
+
 Each `--wire-build-path` / `--ethereum-path` / `--solana-path` flag falls back to
 its env var (`WIRE_BUILD_PATH` / `WIRE_ETH_PATH` / `WIRE_SOLANA_PATH`); one of the
 two is required. `--cluster-path` (env `WIRE_CLUSTER_PATH`) is optional — omit it

diff --git a/packages/flow-swap-epoch-stress/README.md b/packages/flow-swap-epoch-stress/README.md
@@ -0,0 +1,45 @@
+# flow-swap-epoch-stress
+
+Manual-only FlowScenario that reproduces the swap load which exposed the
+Solana terminal/epoch-stall failure. It intentionally multiplies the existing
+Ethereum-to-Solana half of `flow-swap-with-underwriting`:
+
+- 10 distinct prefunded Ethereum actors submit `requestSwap` concurrently.
+- Each actor targets a different Solana recipient.
+- The cluster bootstraps 21 producer accounts and 21 batch operators.
+- One real underwriter is bonded on Ethereum and Solana.
+- The flow verifies all 10 destination payouts, all 10 confirmed UWREQ rows,
+  three post-load epoch advances, and recent Solana outpost logs for
+  memory/heap errors.
+
+Every transaction is its own Step. The generated Markdown, HTML, and CSV
+reports contain the parameters, transaction/RPC evidence, payout observations,
+UWREQ statuses, epoch baseline/result, and relevant Solana log evidence.
+
+## Run locally
+
+Build the required sibling repositories, then use the canonical runner and
+heartbeat monitor in separate terminals:
+
+```bash
+node scripts/run-flow.mjs flow-swap-epoch-stress \
+  --cluster-path /tmp/wire-flow-swap-epoch-stress \
+  --wire-build-path ../wire-sysio/build/debug \
+  --ethereum-path ../wire-ethereum \
+  --solana-path ../wire-solana
+```
+
+```bash
+node scripts/flow-heartbeat-monitor.mjs \
+  --cluster-path /tmp/wire-flow-swap-epoch-stress
+```
+
+This flow can take at least 138 minutes and is excluded from the default E2E
+suite. Run it on demand when validating Solana terminal, validator, OPP, or
+epoch-liveness changes. Reports are written beneath
+`/tmp/wire-flow-swap-epoch-stress/reports/`; cluster logs and OPP debugging
+artifacts remain in the same cluster directory.
+
+The separate `wire-cluster-tool readiness` command remains the read-only tool
+for inspecting an already-running SIM2 cluster; this flow owns and stresses a
+fresh local cluster for reproducible debugging.
diff --git a/packages/flow-swap-epoch-stress/package.json b/packages/flow-swap-epoch-stress/package.json
@@ -0,0 +1,22 @@
+{
+  "name": "@wireio/test-flow-swap-epoch-stress",
+  "version": "0.1.0",
+  "private": true,
+  "type": "commonjs",
+  "description": "Manual FlowScenario: concurrent Ethereum to Solana swaps with epoch-stall diagnostics",
+  "scripts": {
+    "build": "tsc -b tsconfig.json",
+    "test": "node lib/index.js",
+    "clean": "rm -rf lib"
+  },
+  "dependencies": {
+    "@solana/web3.js": "^1.98.0",
+    "@wireio/cluster-tool": "workspace:*",
+    "@wireio/sdk-core": "^1.0.17",
+    "@wireio/shared": "^1.0.17",
+    "ethers": "^6.14.0"
+  },
+  "devDependencies": {
+    "@types/node": "25.5.0"
+  }
+}
diff --git a/packages/flow-swap-epoch-stress/src/SwapEpochStressScenario.ts b/packages/flow-swap-epoch-stress/src/SwapEpochStressScenario.ts
@@ -0,0 +1,257 @@
+import Assert from "node:assert"
+import type { ClusterConfig } from "@wireio/cluster-tool-shared"
+import { SysioContracts } from "@wireio/sdk-core"
+import type { Logger } from "@wireio/shared"
+import {
+  ClusterBuildFailureMode,
+  ClusterBuildPhase,
+  ClusterBuildPhaseGroup,
+  Constants as ClusterConstants,
+  FlowScenario,
+  Report,
+  SwapScenarioContext,
+  WireReserveTool,
+  WireUnderwriterTool,
+  matchesProtoEnum,
+  pollUntil,
+  verifyStep,
+  type ClusterBuild,
+  type ClusterBuildOptions
+} from "@wireio/cluster-tool"
+import { SwapEpochStressScenarioConstants as Constants } from "./SwapEpochStressScenarioConstants.js"
+import {
+  StressBaselineEpochKey,
+  StressBaselineUwreqIdsKey,
+  StressTargetAmountKey
+} from "./SwapEpochStressScenarioOutputs.js"
+import { SwapEpochStressScenarioSteps as StressSteps } from "./steps/index.js"
+
+const { SysioContractName, SysioOpregOperatorstatus } = SysioContracts
+const { Actor } = Report
+
+/** Manual, report-first reproduction for the Solana terminal/epoch-stall load. */
+export class SwapEpochStressScenario extends FlowScenario<SwapScenarioContext> {
+  readonly name = "flow-swap-epoch-stress"
+  readonly description =
+    "Ten concurrent Ethereum→Solana swaps across 21 producers and 21 batch operators, with settlement and epoch-liveness diagnostics"
+
+  override readonly defaults: ClusterBuildOptions = {
+    enableMockReserves: true,
+    epochDurationSec: Constants.EpochDurationSec,
+    producerCount: Constants.ProducerCount,
+    batchOperatorCount: Constants.BatchOperatorCount,
+    underwriterCount: Constants.UnderwriterCount,
+    requiredUnderwriterCollateral: [
+      {
+        chainCode: Constants.EthereumChainCode,
+        tokenCode: Constants.EthereumTokenCode,
+        minimumBond: Constants.UnderwriterMinimumBond
+      },
+      {
+        chainCode: Constants.SolanaChainCode,
+        tokenCode: Constants.SolanaTokenCode,
+        minimumBond: Constants.UnderwriterMinimumBond
+      }
+    ]
+  }
+
+  override createContext(
+    config: ClusterConfig,
+    log: Logger
+  ): SwapScenarioContext {
+    return new SwapScenarioContext(config, log)
+  }
+
+  plan(cluster: ClusterBuild<SwapScenarioContext>): void {
+    const firstUnderwriter = ClusterConstants.underwriterLabel(0)
+    const requestOptions = { timeoutMs: Constants.RequestStepTimeoutMs }
+    const settlementOptions = {
+      timeoutMs: Constants.SettlementDeadlineMs + Constants.PollDeadlineBufferMs
+    }
+    const diagnosticsOptions = {
+      timeoutMs:
+        Constants.postLoadEpochDeadlineMs() +
+        Constants.PollDeadlineBufferMs +
+        120_000
+    }
+
+    WireUnderwriterTool.planCollateralDeposit(
+      cluster,
+      "UnderwriterCollateral",
+      "Bond the underwriter on both outposts before applying swap load",
+      requestOptions,
+      [firstUnderwriter],
+      cluster.context.config.underwriterCollateral ??
+        WireUnderwriterTool.load(null, Constants.UnderwriterCount)
+    )
+
+    ClusterBuildPhase.create(
+      cluster,
+      "StressActors",
+      "Create ten distinct Ethereum senders and ten distinct Solana recipients",
+      Array.from({ length: Constants.ActorCount }, (_, actorIndex) =>
+        StressSteps.planProvisionActor(
+          Actor.User,
+          `stress-actor-${actorIndex + 1}`,
+          `create Ethereum sender ${actorIndex + 1} and Solana recipient ${actorIndex + 1}`,
+          {},
+          actorIndex,
+          Constants.EthereumHdIndexBase + actorIndex
+        )
+      ),
+      {
+        parallelize: true,
+        failureMode: ClusterBuildFailureMode.collect
+      }
+    )
+
+    ClusterBuildPhase.create(
+      cluster,
+      "PreLoadGate",
+      "Require an active underwriter, healthy reserves, and capture the pre-load baseline"
+    ).push(
+      verifyStep(
+        Actor.Underwriter,
+        "underwriter-active",
+        `${firstUnderwriter} is ACTIVE before swaps are submitted`,
+        async ctx => {
+          const account = ctx.keyStore.assertOperator(firstUnderwriter).account
+          await pollUntil(
+            `${firstUnderwriter} ACTIVE`,
+            async () => {
+              const { rows } = await ctx.wire
+                .getSysioContract(SysioContractName.opreg)
+                .tables.operators.query({ limit: 100 })
+              const row = rows.find(operator => operator.account === account)
+              return (
+                row != null &&
+                matchesProtoEnum(
+                  row.status,
+                  SysioOpregOperatorstatus,
+                  SysioOpregOperatorstatus.OPERATOR_STATUS_ACTIVE
+                )
+              )
+            },
+            Constants.underwriterActiveDeadlineMs(),
+            Constants.LongPollIntervalMs
+          )
+        },
+        {
+          timeoutMs:
+            Constants.underwriterActiveDeadlineMs() +
+            Constants.PollDeadlineBufferMs
+        }
+      ),
+      verifyStep(
+        Actor.Sysio,
+        "capture-pre-load-baseline",
+        "verify reserves, compute the live quote, and snapshot epoch/UWREQ IDs",
+        async ctx => {
+          await ctx.reserveBook(
+            Constants.EthereumChainCode,
+            Constants.EthereumTokenCode,
+            Constants.PrimaryReserveCode
+          )
+          await ctx.reserveBook(
+            Constants.SolanaChainCode,
+            Constants.SolanaTokenCode,
+            Constants.PrimaryReserveCode
+          )
+          const targetAmount = await WireReserveTool.swapquote(ctx.wire, {
+            from: {
+              chainCode: Constants.EthereumChainCode,
+              tokenCode: Constants.EthereumTokenCode,
+              reserveCode: Constants.PrimaryReserveCode
+            },
+            fromAmount: Constants.SourceEthereumWei / Constants.WeiPerDepotUnit,
+            to: {
+              chainCode: Constants.SolanaChainCode,
+              tokenCode: Constants.SolanaTokenCode,
+              reserveCode: Constants.PrimaryReserveCode
+            }
+          })
+          Assert.ok(targetAmount > 0n, "ETH→SOL live swap quote is zero")
+          const { rows: epochRows } = await ctx.wire.getEpochState()
+          Assert.ok(epochRows[0], "sysio.epoch::epochstate is empty")
+          const { rows: requestRows } = await ctx.wire
+            .getSysioContract(SysioContractName.uwrit)
+            .tables.uwreqs.query({ limit: 256 })
+          const baselineEpoch = Number(epochRows[0].current_epoch_index)
+          const baselineUwreqIds = requestRows.map(request =>
+            Number(request.id)
+          )
+          ctx.outputs
+            .set(StressTargetAmountKey, targetAmount)
+            .set(StressBaselineEpochKey, baselineEpoch)
+            .set(StressBaselineUwreqIdsKey, baselineUwreqIds)
+          Report.StepExtraRecorder.note("pre-load baseline captured", {
+            targetAmount,
+            baselineEpoch,
+            baselineUwreqIds
+          })
+        }
+      )
+    )
+
+    const stress = ClusterBuildPhaseGroup.create(
+      cluster,
+      "ConcurrentSwapStress",
+      "Apply simultaneous swaps, collect every result, and always run terminal diagnostics",
+      { failureMode: ClusterBuildFailureMode.collect }
+    )
+
+    ClusterBuildPhase.create(
+      stress,
+      "ConcurrentRequests",
+      "Launch ten Ethereum→Solana requestSwap transactions at the same time",
+      Array.from({ length: Constants.ActorCount }, (_, actorIndex) =>
+        StressSteps.planRequestSwap(
+          Actor.User,
+          `request-swap-${actorIndex + 1}`,
+          `Ethereum actor ${actorIndex + 1} swaps to Solana recipient ${actorIndex + 1}`,
+          requestOptions,
+          actorIndex
+        )
+      ),
+      {
+        parallelize: true,
+        failureMode: ClusterBuildFailureMode.collect
+      }
+    )
+
+    ClusterBuildPhase.create(
+      stress,
+      "DestinationPayouts",
+      "Verify all ten distinct Solana recipients receive their remit payouts",
+      Array.from({ length: Constants.ActorCount }, (_, actorIndex) =>
+        StressSteps.planVerifyPayout(
+          Actor.SolanaOutpost,
+          `verify-payout-${actorIndex + 1}`,
+          `Solana recipient ${actorIndex + 1} receives the expected target amount`,
+          settlementOptions,
+          actorIndex
+        )
+      ),
+      {
+        parallelize: true,
+        failureMode: ClusterBuildFailureMode.collect
+      }
+    )
+
+    ClusterBuildPhase.create(
+      stress,
+      "TerminalDiagnostics",
+      "Prove all UWREQs confirmed, the epoch remains live, and Solana logs are clean",
+      [
+        StressSteps.planTerminalDiagnostics(
+          Actor.Sysio,
+          "post-load-epoch-and-solana-health",
+          "require three post-load epoch advances and report UWREQ/Solana memory evidence",
+          diagnosticsOptions,
+          Constants.ActorCount
+        )
+      ],
+      { failureMode: ClusterBuildFailureMode.collect }
+    )
+  }
+}