Skip to content

Run user deletion through saved stage transitions - #8520

Merged
ravarora2 merged 11 commits into
mainfrom
rarora/user-deletion-approval
Oct 26, 2026
Merged

ravarora2 merged 11 commits into
mainfrom
rarora/user-deletion-approval

Conversation

@ravarora2

@ravarora2 ravarora2 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

An operator can submit, approve, and run a synthetic user-deletion request through a durable workflow. The actions print messages; they do not gate real users or delete data. The inventory contains artificial participant and former-participant communities.

The worker calls one transition function, get_next_stage(lease, previous_stage_id). It receives a full stage row, passes it to the handler, then asks again. All branching, community tracking, initialization, and recovery live behind that function. Existing community-deletion execution is unchanged.

Related issue

Builds on merged #8496 and targets main. Inventory, separate approval records, and the dummy executor remain together in this PR.

Execution design

approve → save approval; create no stages
run → claim request
    get_next_stage(lease, None) → full stage row
    handler(lease, row) → persist status and result
    get_next_stage(lease, Some(row.id)) → next row / Finished / WaitUntil
    repeat, then release ownership
  • Approval: bind an operator's decision to the saved inventory digest. Approval creates no execution rows. Operator names remain self-asserted demo input.
  • Initialization: claiming preserves the parent lifecycle. The first transition creates gate_user and changes approved to running atomically. A crash after claim but before initialization can safely restart.
  • Handler: read metadata_json.inputs, mark the row running, and save its result or failure. A successful action leaves its row completed and is_current = true. This is a durable transition waiting to be processed.
  • Transition: lock the parent, validate ownership, reload history and the handler's saved result, and choose an allowed destination. Clear the old current flag, save the decision, and insert the successor with its inputs in one transaction. Terminal transitions complete the parent instead. An insertion failure leaves the completed old row current.
  • Recovery: previous_stage_id identifies the handler that just returned; it must belong to the request. A restarted worker passes None and finds the saved current row. If that row is completed, transition it without repeating its action. If the switch already committed, return the saved successor. Repeated calls never apply the old transition twice.
  • Progress: reconstruct covered communities from saved decisions and the approved inventory. Each stage records its predecessor ID, so reconstruction does not depend on timestamp ordering. Missing current state in a started request, disconnected history, unknown stages, and malformed results fail without inventing progress.
  • Ownership: parent leases last 60 seconds and handlers renew them every ten seconds. Generation checks reject stale writes. Handler and renewal futures remain independently polled; renewal cannot prevent a handler holding the parent lock from committing.
  • Step-by-step testing: run --next-step executes one handler, commits its transition, and exits with the successor pending. Repeat the same command to advance again; the final step completes the request. The former --max-stages and --pause-after-handler flags are removed.
  • Failure: the handler saves the current row's error and a 30-second retry deadline. The transition returns WaitUntil; the worker exits and releases its lease. A later invocation retries the same row. No background scheduler or automatic retry daemon is added.

The user workflow definition in buzz-deletion/src/user/workflow.rs declares allowed destinations and a decision function per stage. gate_user enters the first uncovered community. purge_db reads saved has_agents: true selects purge_agents; false completes the community immediately. Community completion selects the next uncovered community, or finishes the request when all are covered. No full stage plan is created upfront. The transition transaction contains no stage-name comparisons or fixed community count.

Stage metadata stores inputs, result, transition, and previous_stage_id. The parent retains inventory, ownership, retry, and overall status. Approval evidence stays in entity_deletion_approvals. A completed-current row bridges the two transactions; real handlers will still need idempotent or reconciled actions if a process dies before saving its result.

Code organization

buzz-deletion/src/user/worker.rs runs the loop and heartbeat. Sibling handlers.rs executes dummy actions. transition.rs defines UserDeletionTransition.get_next_stage(), and workflow.rs owns the user stage rules and community-progress reconstruction. The CLI remains in user.rs.

buzz-db/src/store/entity_deletion/ contains shared workflow persistence: a small mod.rs, request reads/inspection in requests.rs, approval evidence in approvals.rs, ownership in leases.rs, and stage outcomes/atomic transitions in stages.rs. User-specific submission, demo inventory, and inventory validation live separately in buzz-db/src/store/user_deletion/{requests,inventory}.rs; its mod.rs only defines and exports the store.

The worker calls the user transition object, which supplies a synchronous decision function to EntityDeletionStore.transition_stage. The store locks the parent, reads saved state, invokes that function, checks the returned operation, and commits its writes. The decision function performs no SQL or external calls. Reading, deciding, and writing remain in one transaction; there is no stale-state gap. Shared storage contains no user stage names or community-routing rules. User-specific approval validation also runs inside the shared approval transaction.

This reorganization changes no tables, migrations, stage metadata, or CLI behavior. Requests using the existing saved-result format continue to work. --next-step still runs one handler and commits its transition before returning.

Testing

The walkthrough in docs/operator-user-deletion-executor.md includes commands, SQL inspection, expected rows, and recovery boundaries.

Submitted and approved a fresh local request and verified zero stage rows. Ran run --next-step five times. Each call completed exactly one handler, released ownership, and left its successor pending: B DB, B agents, C DB, C agents, then request completed. A sixth call left saved state unchanged. Verified both removed flags are rejected.

Ran --demo-no-agents-in 00000000-0000-0000-0000-00000000000c and verified four completed rows, with no C agents stage.

Injected a DB-stage failure, verified immediate retry refusal, waited for the real deadline, and resumed the same stage row. Observed heartbeat renewal and rejection of a competing worker, killed the owned worker process, waited for its actual lease expiry, and resumed the same unfinished stage with a new generation. Re-running a completed request left its saved state unchanged. Repeated the CLI walkthrough after the file reorganization, including runtime branching and a retry after the real deadline; a completed request created before the reorganization remained unchanged.

Mutation-tested the ownership and recovery guards against local PostgreSQL 17. Reclaiming with the same executor ID rejects old-generation operations. Independently disabling the ownership generation check, release generation check, saved-transition/result comparison, or heartbeat-failure handling made the corresponding test fail. Restored all four protections afterward. The heartbeat test takes over the request while its handler is already running.

Schema and rollout

Unmerged migrations 0069–0070 add inventory and execution fields, widen request lifecycle states, and create operator-global approval and stage-history tables. They set five-second lock and thirty-second statement timeouts; constraint validation has its own transaction. No triggers, functions, foreign keys, or data backfills are added. Desired-state and migration catalog parity cover all three tables. The transition redesign adds no further schema changes.

The serving relay does not schedule these requests. Only the explicit operator CLI runs them, using PostgreSQL. Only demo: targets are accepted. Real account discovery, user-write fencing, owned-community handling, and actual deletion remain future work. Verify staging with a new demo label and the documented commands.

Older draft requests do not have the new stage metadata contract. Preserve them and create a new demo request for this version; they are not a supported resume format. Use a fresh database if an earlier draft applied a different migration checksum. Never alter sqlx migration checksums. With Helm migrate.autoMigrate=true (the chart default), a relay binary that lacks 0069/0070 fails at boot with sqlx VersionMissing after those migrations are applied. Recovery for this rollout is a roll-forward: deploy a corrected binary that retains the migrations. Do not delete migration records or change checksums to force an older binary to boot. Rolling back with auto-migration disabled requires a separately verified schema-compatibility plan. Demo requests, stage history, and approvals are intentionally retained; this scaffold has no cleanup command or retention job. Recheck migration numbering before merge.

Generated with Codex

@ravarora2
ravarora2 force-pushed the rarora/user-deletion-approval branch from 2611142 to 7f1f0bf Compare October 21, 2026 21:44
@ravarora2
ravarora2 force-pushed the rarora/BUZZ-301/user-deletion-submit branch from 104a4ff to 5cde354 Compare October 21, 2026 21:44
Base automatically changed from rarora/BUZZ-301/user-deletion-submit to main October 22, 2026 14:49
Signed-off-by: Ravneet Arora <rarora@squareup.com>
Signed-off-by: Ravneet Arora <rarora@squareup.com>
Signed-off-by: Ravneet Arora <rarora@squareup.com>
@ravarora2
ravarora2 force-pushed the rarora/user-deletion-approval branch from 109b0a0 to 4727351 Compare October 22, 2026 15:29
@ravarora2 ravarora2 changed the title Add demo user-deletion inventory and approval Run resumable dummy user deletion workflows Oct 22, 2026
Signed-off-by: Ravneet Arora <rarora@squareup.com>
@ravarora2 ravarora2 changed the title Run resumable dummy user deletion workflows Run user deletion through saved stage transitions Oct 22, 2026
Signed-off-by: Ravneet Arora <rarora@squareup.com>
Signed-off-by: Ravneet Arora <rarora@squareup.com>
Signed-off-by: Ravneet Arora <rarora@squareup.com>
@ravarora2
ravarora2 marked this pull request as ready for review October 23, 2026 16:20
@ravarora2
ravarora2 requested a review from a team as a code owner October 23, 2026 16:21
@ravarora2
ravarora2 deployed to codex-review October 23, 2026 16:21 — with GitHub Actions Active
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 23, 2026
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Note: This is an automated, security-focused review generated by Codex.
Use it as a supplement to human review; false positives are possible.

Scope

  • Exact PR diff: 9e8c9fc04daf20259cd259dcaec89109587cc7f5...4690264b2289d0c9bfcd6a488fc87cbf94778226
  • Model: gpt-6.1-sol

💡 Click "edited" above to see earlier reviews for this PR.


Review Summary

Overall Risk: NONE

No concrete security, correctness, or reliability findings identified in the authorized PR range.

Findings

No concrete security, correctness, or reliability findings were identified.

Notes

  • Review used read-only source inspection. Builds and tests were not executed.

Generated by Codex Security Review |
Requested by: @ravarora2 |
Workflow run

Signed-off-by: Ravneet Arora <rarora@squareup.com>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 23, 2026
@ravarora2
ravarora2 deployed to codex-review October 23, 2026 16:31 — with GitHub Actions Active
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 23, 2026
Signed-off-by: Ravneet Arora <rarora@squareup.com>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 23, 2026
@ravarora2
ravarora2 deployed to codex-review October 23, 2026 19:10 — with GitHub Actions Active
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 23, 2026
Signed-off-by: Ravneet Arora <rarora@squareup.com>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 23, 2026
@ravarora2
ravarora2 deployed to codex-review October 23, 2026 19:17 — with GitHub Actions Active
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 23, 2026
Signed-off-by: Ravneet Arora <rarora@squareup.com>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 23, 2026
@ravarora2
ravarora2 deployed to codex-review October 23, 2026 19:30 — with GitHub Actions Active
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Oct 23, 2026
@ravarora2
ravarora2 merged commit 4250014 into main Oct 26, 2026
147 of 150 checks passed
@ravarora2
ravarora2 deleted the rarora/user-deletion-approval branch October 26, 2026 14:44
TheSentinel454 added a commit that referenced this pull request Oct 26, 2026
Renumber the unlisted migration to 0071: main took 0069 and 0070 for
entity deletion (#8520).

Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Luke Tornquist <tornquist@squareup.com>
ravarora2 added a commit that referenced this pull request Oct 26, 2026
)

## Summary

User deletion can now discover current communities from a hex public key
or npub, save that inventory, and run it through the existing approval
and dummy workflow. `--pubkey demo` remains the default. Execution only
prints messages; it does not fence users or delete data.

Discovery combines explicit relay memberships with active channel
memberships, including users who have no relay membership row. It
deduplicates communities and preserves owner relationships. The saved
manifest contains only community IDs, labels and relationships.
Historical discovery and owned-community execution remain later work.
Submission rejects owner inventories atomically with a specific
unsupported-execution error; preview still includes owners.

This is an expand-schema change after #8520. Migrations 0072–0073
replace the demo-only target check with a bounded identifier check, with
5-second lock and 30-second statement timeouts. The database keeps the
user-only entity-type check while allowing different identity formats in
the target field. Workflow-specific commands reject entity types they
cannot execute. The application currently resolves demo and hex/npub
inputs; account-ID lookup is future work. The desired-state constraint
matches. No backfill, trigger, function, or foreign key is added.
Existing demo rows remain valid. Old binaries fail closed on new
manifests; use the new CLI for their approval and execution. Older
auto-migrating binaries need the documented rollback procedure.

Identity parsing and demo fixture selection stay in intake. Approval and
resume use the saved target without reparsing its format. The user
workflow defines its stage names and handlers; CLI stage validation
reads those rules instead of a separate three-name list. Shared
persistence accepts caller-defined stages. Permanent target uniqueness
is unchanged. Dummy submissions and execution belong in isolated,
disposable development databases; production preview uses read-only
access. CLI operator names remain self-asserted.

### Related issue

Follows #8520. Searched open PRs; no duplicate inventory change found.
#8489 is earlier workflow scaffolding.

### Testing

Built `buzz-admin` locally and applied all migrations to a fresh
disposable PostgreSQL database. Ran the actual CLI through preview →
submit → inspect → approve → `run --next-step` → resume → completed for
both demo and a public key with a channel-only membership fixture. Demo
completed five stages; the public-key run completed three. Injected
`--fail-stage purge_db`, waited for the retry deadline, and resumed the
same failed stage successfully. Membership data remained intact.

Repeated the CLI workflow on October 26 for demo and an npub backed by a
fresh production inventory. Production access used read-only credentials
and the real `inventory --pubkey "$SUBJECT"` handler. Copied only the
relevant discovery fields into an isolated local PostgreSQL database;
local channel parent rows supplied placeholder fields for foreign keys.
The inventory saved by local submission exactly matched the production
result. All workflow writes stayed local.

For each input, ran `submit --pubkey "$SUBJECT" --requested-by
local-e2e-submitter`, inspected the saved inventory, and confirmed
execution before approval was rejected. Approved its digest, then ran
each handler with `run "$REQUEST_ID" --next-step --stage-delay-ms 2500`
in a new process. Captured all three workflow tables during and after
every stage. Demo completed five stages across two communities; the npub
completed three stages for its discovered community. Each handler saved
`dummy: true`. Both requests finished with no current stage, no worker
lease, and no retries. Copied memberships remained intact. This
additional run exercised orderly resume; it did not inject a failure.

The reproducible command sequence and fixture setup are in [the
inventory walkthrough](docs/operator-user-deletion-inventory.md).
Production was not changed. The earlier full-workflow runs used
synthetic local fixtures; the additional npub run used captured
production discovery fields. Repository-wide `just ci` was not run;
validation focused on the affected packages and PostgreSQL workflow. The
pre-push desktop build failed with “Too many open files in system.”
Stopped the broader hook run and bypassed it to publish the earlier
commits; the affected-package Clippy and formatting checks had passed.

Generated with Codex

---------

Signed-off-by: Ravneet Arora <rarora@squareup.com>

This branch was successfully deployed

1 active deployment
codex-review — 4690264b Deployed Oct 23, 2026 by ravarora2 via Run Codex Security Review #9676
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-security-review-current The posted Codex security review matches its recorded range.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants