Skip to content

fix(sysio): preserve epoch batch reward rosters (WIRE-343) - #576

Open
huangminghuang wants to merge 3 commits into
masterfrom
fix/wire-343-multi-epoch-batch-operator-rewards
Open

fix(sysio): preserve epoch batch reward rosters (WIRE-343)#576
huangminghuang wants to merge 3 commits into
masterfrom
fix/wire-343-multi-epoch-batch-operator-rewards

Conversation

@huangminghuang

@huangminghuang huangminghuang commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Record the batch-operator roster for every accrued epoch and pay rewards from immutable per-epoch history, preserving roster identity through schedule rotation.
  • Keep the public payepoch ABI compatible while moving roster input to the new batchepochs table.
  • Bound new cadence writes to 1–10 epochs; clamp legacy stored values at the epoch gate; prune legacy overlong history without halting advance.
  • Recover safely from missing, stale, non-contiguous, or over-cap history by retaining that period’s batch-emission and swap-fee slices in treasury, clearing unusable history, and resuming cleanly next period.
  • Refresh deployable WASM/ABI artifacts and align the emission and upgrade documentation with the bounded recovery behavior.

Validation

  • Workflow runner: hygiene, CDT SDK freshness, configure, full build, and contracts_unit_test -- --sys-vm passed (660 tests).
  • CI-equivalent artifact-copy configuration: build and contracts_unit_test passed (660 tests); checked-in WASM/ABI artifacts byte-match the build outputs.
  • Canonical local flow-emissions-soak passed: 81 phases, 311 steps, 2,176s; its heartbeat monitor saw no fatal signatures.
  • Independent correctness, architecture, and compliance reviews passed.

Pending

  • GitHub CI and the targeted remote platform-flow receipt are pending against head 619f5d0a5a2ada1f17449f994009cd531d8ee422.

Refs: WIRE-343

Change-Id: Iaa60f6d7cfad4dc0662b5b81b6e8781acf120741
@huangminghuang
huangminghuang requested a review from a team August 20, 2026 00:29
@heifner heifner removed their assignment Aug 20, 2026
@heifner
heifner self-requested a review August 20, 2026 13:29

@heifner heifner left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design looks sound — the roster recorded by rcrdbatch is read after the window slide so it matches the epoch/group accrueepoch credits, the KV key is big-endian so batchepochs iterates in ascending epoch order (contiguity walk is valid), the erase(it) loop is safe, and ABI table_id 22503 has no collisions.

5 findings inline: 2 chain-halt risks on the mandatory advance inline path, 1 missing build artifact, 2 minor.

++history_count;
}
sysio::check(history_count < emissions::MAX_PAY_CADENCE_EPOCHS,
"batch roster history exceeds safety cap");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium — chain halt. rcrdbatch runs inline from sysio.epoch::advance, so this throw aborts the whole advance transaction and halts epoch advancement chain-wide.

A chain whose stored emitcfg.pay_cadence_epochs is still >10 (the header comment this PR deletes recommended 100) accrues fine for epochs 1–10, then aborts on the 11th advance — and every retry after it. It is not self-recoverable: setemitcfg's new <= 10 bound only applies on write, and lowering the cadence afterwards doesn't help because rcrdbatch throws before payepoch (which clears the history) can ever run.

accrueepoch's own comment two functions up states the rule: "this must never throw — a throw would abort epoch advancement chain-wide." Prefer clamping/pruning here, or validating the cadence where it is read.

&& recorded_epochs == accrued_epochs
&& expected_epoch_index == static_cast<uint64_t>(epoch_index) + 1;
sysio::check(accrued_epochs == 0 || batch_history_complete,
"batch roster history incomplete");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium — chain halt. An incomplete history hard-aborts payepochadvance → chain-wide epoch stall, with no degraded path.

Every mixed-state deployment trips it:

  • new sysio.system + old sysio.epoch (no rcrdbatch sent → history empty)
  • deploying mid-period (rows start at the deploy epoch, expected_epoch_index starts at period_start_epoch)
  • a legacy t5state.batch_group_epochs with non-zero counters from before activation
  • a stale batchepochs row surviving a period

docs/contract-upgrade-order.md mitigates by procedure only — one missed operator step bricks the chain. Contrast sysio.epoch.cpp:594, which prints a warning for an empty new-tail group instead of throwing, precisely because "aborting advance() here would halt OPP epoch advancement chain-wide." A print-and-retain-in-treasury fallback for the first period would remove the cliff.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more trigger for this same abort, from the seed two lines above the scan (emissions.cpp:793):

uint64_t expected_epoch_index =
   state.period_start_epoch == 0 ? 1 : state.period_start_epoch;

The == 0 fallback hard-codes "the first accrued epoch is 1". A chain that runs initt5 while sysio.epoch is already past epoch 1 records its first roster at the current epoch, the contiguity walk starts expecting 1, batch_history_complete goes false on the very first row, and the check here aborts payepochadvance permanently — the same non-recoverable stall as the other mixed-state cases, but reachable on a clean activation rather than a partial upgrade.

Seeding from the first recorded row (or from state.last_epoch_index at initt5 time) instead of the literal 1 removes this one; the degraded-path suggestion above still covers the rest.

"key_types": ["uint64"],
"table_id": 49446
},
{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium — artifact mismatch. The ABI is refreshed (rcrdbatch, batch_epoch/batch_epoch_key, batchepochs) but neither sysio.system.wasm nor sysio.epoch.wasm is in this PR.

The committed source-tree artifacts are now inconsistent: the ABI advertises an action the committed WASM cannot dispatch, and the sysio.epoch behavior change ships no artifact at all. Anything reading the source-tree pair (generate-sysio-contract-types.py -B ., or a deploy that doesn't build first) sees a surface that doesn't match the code.

Prior changes committed the pair together — 8638612f08, 778f86896c, 5f90047009.


// A pay period is the history lifetime. The whole action is atomic, so this
// can only clear a complete history after its corresponding payout.
for (auto it = batch_history.begin(); it != batch_history.end(); ) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low. This clear loop sits outside the if (accrued_epochs > 0) guard just above, and the completeness check at L823 also short-circuits when accrued_epochs == 0.

So on the zero-accrual path — guarded as "impossible in practice" but defended against everywhere else in this function — payepoch erases every recorded roster without paying any of them: the emission and fee slices are silently retained and the rosters' identity is destroyed rather than carried into the next period. Moving the erase inside the accrued_epochs > 0 branch keeps the two paths consistent.

).send();

if (gate.is_pay_epoch) {
action(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low. payepoch's second parameter is now unnamed/unused (emissions.cpp:727), but advance still serializes state.batch_op_groups into it on every pay epoch — up to MAX_SCHEDULED_BATCH_OPERATORS = 1,000 names of dead payload on a critical inline action.

The ABI field can't be dropped without a coordinated change, but consider sending an empty vector, or noting the intent so a future reader doesn't restore a use for it.

Change-Id: I0cb745aebe72c6b63c0bc7bbe0af94254b27dd52

@heifner heifner left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 1ab98aff5 (after the artifact refresh). Status of the 2026-08-20 findings, plus one new one.

Resolved

  • sysio.system.abi:2124 — artifact mismatch. 1ab98aff5 adds both sysio.system.wasm and sysio.epoch.wasm, so the source-tree pair is consistent again. One nit for the squash: that commit touches only the two .wasm files with no accompanying source change, which is the shape commit-wasm-only-with-source-change.md bans per-commit. The PR as a whole pairs source with artifacts, so this is history hygiene rather than a provenance gap.

Still open, unchanged at head

  • emissions.cpp:684rcrdbatch's cap throw on the mandatory inline advance path.
  • emissions.cpp:823 — incomplete-history hard abort. One more concrete trigger added in that thread.
  • emissions.cpp:1043 — the clear loop is still outside the accrued_epochs > 0 guard: the guard opens at L1035 and closes at L1039, and batch_history.erase runs unconditionally at L1043.
  • sysio.epoch.cpp:889payepoch's now-unused second parameter is still serialized on every pay epoch.

New

  • emissions.cpp:678 — an O(cadence × roster) deserialize on every advance, used only to produce a count. Inline below.

const batch_epoch_key key{epoch_index};
sysio::check(!history.contains(key), "batch roster already recorded for epoch");
uint16_t history_count = 0;
for (auto it = history.begin();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low — efficiency, on the epoch-boundary path. This loop exists only to produce a count, but iterating batchepochs_t fully deserializes every row it visits — up to MAX_PAY_CADENCE_EPOCHS (10) batch_epoch rows, each carrying a members vector holding that epoch's whole serving group — and it runs on every advance, not only on pay epochs.

The count is never used except in the < MAX_PAY_CADENCE_EPOCHS comparison two lines down, and the invariant it guards ("history never outlives one period") is already implied by payepoch clearing the table at each period boundary. Two O(1) equivalents:

  • carry the count on t5state, next to the existing per-epoch bookkeeping — incremented here, zeroed where the history is cleared at emissions.cpp:1043; or
  • probe only the row that would push the table over the cap — history.contains(batch_epoch_key{epoch_index - emissions::MAX_PAY_CADENCE_EPOCHS}) — which needs no deserialize, but wants an epoch_index > MAX_PAY_CADENCE_EPOCHS guard against the uint32_t underflow and leans on the same contiguity the completeness check at L818 already assumes.

Worth doing because this is the same transaction whose CPU budget accrueepoch's comment two functions up is explicitly protecting — paying an O(cadence × roster) deserialize per epoch for a bound that is checkable in O(1) is the wrong trade on that path.

Change-Id: Ic6236e59b9d59eb8727d55228ceaea2d9b8aee86
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants