fix(din): clock-skew partition clamp + WriteTimeout (telematics-scale review) - #9
Open
zer0stars wants to merge 1 commit into
Open
fix(din): clock-skew partition clamp + WriteTimeout (telematics-scale review)#9zer0stars wants to merge 1 commit into
zer0stars wants to merge 1 commit into
Conversation
…ut > ack budget (scale review) - #1 broken-clock partition pollution: raw_events is PARTITIONED BY day("time"), so a device with a garbage RTC (epoch-0/1970, 1980/2019 GPS-week rollover, 2000 factory default, 2099 runaway) minted a permanent singleton day-partition maintenance can never merge — an unbounded partition/file/catalog leak once even a small fraction of a large fleet has bad clocks. partitionSafeTime clamps a time outside [now-365d, now+24h] to now. IMPORTANT (adversarial-review fix): the clamp is applied ONLY to the value WRITTEN to raw_events (fillRowArgs, the day("time") partition key), NOT to the CloudEvent header time. An earlier version clamped the header time in convert, which changed the NATS MsgID / the decodestream dedup id (both hash the header time) and made a stable-id broken-clock device's retries drift to new MsgIDs → duplicate raw rows → double-fire to vehicle-triggers, plus a sibling-collapse data-loss edge. Clamping only the stored partition value bounds the partition while leaving retry dedup and sibling identity exactly as before. Wide bounds keep legit offline buffers and near-future skew untouched. now is computed once per bundle. - #5 WriteTimeout(5s) < publishAckTimeout(10s): the handler blocks up to the ack budget before mapping a lost ack to 503+Retry-After, which the socket WriteTimeout cut short → the device saw a raw reset and retry-stormed under burst latency. Split WriteTimeout from the read timeout (DefaultWriteTimeout 15s, connection = publishAckTimeout+margin). The ATTESTATION server additionally budgets its pre-publish ERC-1271 verify (5s) + blob PUT (attestationPrePublishBudget) on top of the ack budget so its 503 is writable too. Tests: partitionSafeTime bounds; rowArgs clamps the STORED time but leaves the header time untouched; server write budget exceeds read + ack budget.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Telematics-scale fixes (din) — #1 clock-skew partition clamp, #5 WriteTimeout
From the telematics-at-scale review (
REVIEW-din-dq-vt-scale-2026-07-08.md), the two din items, hardened after an adversarial regression review.#1 — broken-clock partition pollution
raw_eventsisPARTITIONED BY day("time"), so a device with a garbage RTC (epoch-0/1970, 1980/2019 GPS-week rollover, 2000 factory default, 2099 runaway) minted a permanent singleton day-partition that maintenance can never merge — an unbounded partition/file/catalog leak once even a small fraction of a large fleet has bad clocks.partitionSafeTimeclamps a time outside[now-365d, now+24h]tonow.Applied at the write layer, not convert (adversarial-review fix): the clamp touches ONLY the value written to
raw_events(fillRowArgs, the partition key), not the CloudEvent header time. An earlier convert-layer version clamped the header time, which changed the NATS MsgID + decodestream dedup id (both hash the header time) → a stable-id broken-clock device's retries drifted to new MsgIDs → duplicate raw rows → double-fire to vehicle-triggers, plus a sibling-collapse data-loss edge. Clamping only the stored partition value bounds the partition while leaving retry dedup and sibling identity exactly as before. Wide bounds leave offline buffers (days-weeks) and near-future skew untouched.nowis computed once per bundle.#5 — WriteTimeout(5s) < publishAckTimeout(10s)
The handler blocks up to the ack budget before mapping a lost ack to
503+Retry-After, which the 5s socket WriteTimeout cut short → the device saw a raw connection reset and retry-stormed under burst latency. Split WriteTimeout from the read timeout (DefaultWriteTimeout15s, connection =publishAckTimeout+margin). The attestation server additionally budgets its pre-publish ERC-1271 verify (5s) + blob PUT on top of the ack budget so its 503 is writable too.Verification
go build/vet/test ./...(18 pkgs ok) /golangci-lint(0 issues). Tests:partitionSafeTimebounds;rowArgsclamps the stored time but leaves the header untouched; server write budget exceeds read + ack budget. Adversarial regression review (4 lenses) confirmed no MsgID/dedup regression after the write-layer move.🤖 Generated with Claude Code
https://claude.ai/code/session_01SAgMMjdGVCD7M1yLy7BUPq