Skip to content

feat(stack-encrypt): chained plan builder as the single front door for writes, queries and reads - #1071

Merged
coderdan merged 10 commits into
feat/stack-encrypt-engine-plan-piecesfrom
feat/stack-encrypt-plan-builder
Oct 5, 2026
Merged

coderdan merged 10 commits into
feat/stack-encrypt-engine-plan-piecesfrom
feat/stack-encrypt-plan-builder

Conversation

@coderdan

@coderdan coderdan commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Stacked PR. This is the third of four PRs. It is based on #1069, which is itself based on #1068, so until those merge the diff below includes their commits. Once they merge, this PR shows only its own commit.

Summary

stack-encrypt is our Rust library for field-level encryption, with keys from ZeroKMS, our key management service. Until now a caller drove its engine by composing low-level building blocks ("combinators") by hand, and a search query had to restate a field's context by hand too. If the query spelled the context differently from the write, it matched nothing, and nothing said so.

This PR adds one chained plan builder: cipher.encrypt(&value).context("users").fields()...await. A plan is that chain saved without the value. The write, the query and the read all take the same plan, so they cannot disagree about a field's context, and a query against an index the field never declared is an error instead of an empty result. Every chain call lowers to an existing combinator; nothing new touches a key or adds cryptography.

Changes

The chain (stack_encrypt::plan, new module)

  • StackCipher::encrypt(&v) starts a chain. .context(c) seals one value under one label; .with(indexes) adds search indexes beside it; .fields() seals each top-level field under <context>/<field>.
  • Field verbs: encrypt(name), encrypt_index(name, indexes), index(name, indexes) (indexes alone, no ciphertext), passthrough(name) (carried unencrypted and unauthenticated), plus identity(seg) to pin the segment a renamed field is keyed under.
  • .keyset(..) (a keyset handle, id or name) and .extend(parts) (extends every field's context for one call) go anywhere in the chain. .await finishes it through IntoFuture. Before the await nothing has touched a key; Operation::prepare shows the chain yields a Pending synchronously.

Saved plans

  • Plan::context(c).fields()...build()? is the chain without a value. build() refuses: a field named twice, a label that is not plain, two fields sharing one identity, passthrough on an indexed field, an index named twice, and, when the value type declares a schema, a field the type lacks, a type field the plan leaves unnamed, or a field at the wrong type. A value is checked against the plan again every time it runs.
  • Plan::context(c).with(indexes).build()? is a one-value plan (ValuePlan).
  • cipher.encrypt(&user).using(&users_plan) runs a plan over a value, a slice or a Vec. A collection settles every key request in one ZeroKMS call.
  • A plan is data: Clone, Debug, reusable. Plan::encryption() and Plan::decryption() expose the lowered descriptions for KeysetCipher::run / run_decryption.

Queries and reads

  • cipher.query(v).using(&users_plan.field("email")?).equality() (or .index(Ore)) derives a search term under exactly the label the write used. An undeclared index is PlanError::IndexNotDeclared; a query value of the wrong type is PlanError::FieldType. On a one-value plan the index is selected by type through Indexes::select, so an undeclared index does not compile.
  • cipher.open(row).using(&plan) decrypts the fields that can come back (sealed and passthrough fields, not index-only ones).
  • stack_encrypt::all((a, b)) settles two to four chains in one ZeroKMS request, under one keyset.

Engine and errors

  • Two crate-internal combinators in target/operations.rs: project_by (pick a field by name at run time, which may fail) and inspect (check the source without I/O). Both are pub(crate); the public project is unchanged.
  • Error::Plan(PlanError) carries every refusal. All are raised before any key is requested.

Value shape

Verification

All run in the worktree on the pinned Rust 1.94.1:

  • mise x --env test -- cargo nextest run --workspace --all-features: 837 tests, all passed (includes the trybuild ui suite with the three new compile-fail cases and one new must-compile case).
  • mise run test:doc: passed, including the four new doctests on the plan module and all.
  • mise run doc (rustdoc, warnings are errors): passed.
  • cargo fmt --all --check: clean.
  • cargo clippy --locked --no-deps --workspace --all-targets --all-features -- -D warnings (what CI runs): clean. Also cargo clippy --target wasm32-wasip1 -p stack-encrypt --no-default-features --features dynamic -- -D warnings: clean.
  • cargo check --target wasm32-wasip1 -p stack-encrypt --no-default-features, and cargo test / cargo doc with --no-default-features (the wasm:no-http-test shape): passed.
  • cargo mutants -p stack-encrypt --in-diff <this PR's diff>: first run 213 mutants, 5 missed (four chains whose named keyset was never checked, and one boolean in the duplicate-field rule). Tests were added in the second commit; re-running those 38 mutants: 14 caught, 24 unviable, 0 missed.
  • mise run wasm:guest:test (the Go binding's WebAssembly guest): 50 tests passed.
  • cargo test -p eql-bindings --features stack-encrypt (from packages/eql): passed.

What the new tests in tests/plan.rs prove:

  • The chain for a four-field record (email: Equality + Match, age: Equality + ORE, notes: encrypt only, id: passthrough) under context users produces the same terms, byte for byte, as the hand-written combinators using the derive's spelling of each field's context. ZeroKMS is asked for keys under the same descriptors (users/email, users/age, users/notes). Each spelling's ciphertext opens under the other's context. Ciphertexts are randomised, so they cannot be compared byte for byte.
  • A write followed by a query gives equal term bytes for equality, match and ORE, including when the call context is extended (.extend(7u64)), and a different extension does not match.
  • Every build() error, every run-time value or row mismatch, and every query refusal has its own test. Each asserts that no key request was made.
  • all((..)), and a plan over a slice or Vec, make exactly one generate call (and one retrieve call when decrypting), counted with the existing counting key source.

Related

Review notes

Start with src/plan/mod.rs (worked examples and the lowering table), then src/plan/build.rs (plans, validation, lowering) and tests/plan.rs.

Departures from the plan document, each forced or deliberate:

  1. The decrypt entry is cipher.open(row), not cipher.decrypt(row). StackCipher::decrypt(ct, aad) is the existing cipher-directed decrypt, with callers across the repo (protect-ffi, the Go guest, tests). Rust has no overloading, so a one-argument decrypt would break all of them. encrypt and query had no clash: StackCipher had neither, and KeysetCipher::encrypt(v, aad) is untouched.
  2. Field verbs name the field type where Rust cannot infer it: .encrypt_index::<u32>("age", (Equality, Ore)). Rust cannot learn a field's type from a string. A value whose fields are all one type (a dynamic map) needs no turbofish. The derive writes the turbofish for its users.
  3. A one-value plan needs .build()?, like a fields plan, so validation has one place.
  4. Open question settled: a one-value plan carries its plaintext type (ValuePlan<S, X>), so opening through it yields S.
  5. A fields plan's type names the data-key source, Plan<S, K>. The descriptions a plan produces are generic over K, and Rust cannot store a method that is generic over it. K is inferred where the plan is run.
  6. A field plan answers queries by index data, a one-value plan by type. A field of a fields plan holds its indexes as data once built, so .equality() there checks the declared IndexSpecs at run time. A ValuePlan keeps its index types, so it uses Indexes::select.
  7. No owned mode. Chains borrow the value, and each field is cloned once where its operation consumes it, as the derive does today. An owned fields plan needs a combinator that splits a record into its fields without cloning the rest, which is follow-up work.
  8. Not in this PR: the JSON index and encrypt_into::<Domain> field targets (later PRs in the plan), and a typed output struct (the derive's job).
  9. One derive snapshot changed by path qualification only. FieldKind::Encrypt makes rustc spell the trait stack_encrypt::Encrypt in tests/ui/plaintext_needs_encrypt.stderr. The derive itself is untouched.

For #1058 (the derive emits the builder). A derived record type must implement Fields (field_names, and schema() so build() checks the plan against the type) and Field<F> for each distinct field type. T::plan() should return Result<Plan<Source, K>, Error> built as Plan::context(<ctx>).fields().<verb>::<FieldType>(name, indexes)...build(); EncryptFrom::encryption() is then plan.encryption().try_map(|mut values| Ok(Target { email: values.take("email")?, .. })), and DecryptInto builds a FieldValues from the stored struct and runs plan.decryption(..). Field labels equal the derive's nonempty!(ctx).with(field) spelling (the byte-identity test checks this).

Glossary/ADR. Nothing here needs a change to the files on #1052, except that the plan document's "decrypt" call is spelled open (point 1) and its one-value plan has a build() (point 3).

@changeset-bot

changeset-bot Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 587be2e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Mutation testing (cargo-mutants, --in-diff, stack-auth + stack-encrypt)

caught missed unviable timeout
200 0 200 0

Every mutant in the changed lines was caught by a test.

Comment thread packages/stack-encrypt/src/plan/build.rs Outdated
Comment thread packages/stack-encrypt/src/plan/build.rs
Comment thread packages/stack-encrypt/src/plan/chain.rs
Comment thread packages/stack-encrypt/src/plan/error.rs
Comment thread packages/stack-encrypt/src/plan/chain.rs Outdated
Comment thread packages/stack-encrypt/src/plan/error.rs
Comment thread packages/stack-encrypt/src/plan/values.rs Outdated

@coderdan coderdan left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of #1071 at 49384663a (this PR's two commits only, diffed against feat/stack-encrypt-engine-plan-pieces)

Comment-only review. No blocking findings. Each item has an inline comment with the detail and, where a scenario was probed, the probe.

Should fix

  1. packages/stack-encrypt/src/plan/build.rs:630 — a pinned identity does not rescue a non-plain field name. build() checks name for plainness even when identity(..) is set, though the name never enters a label then (it is only the record key and the Fields lookup string). Probe at this commit: .encrypt::<String>("2fa_secret").identity("totp_secret").build() → FieldLabel for "2fa_secret", while the only label built is users/totp_secret. That is the column-rename case identity exists for.
  2. packages/stack-encrypt/src/plan/build.rs:989 — a Vec written through a one-value plan cannot be opened through it. Runs<[S]> / Runs<Vec<S>> exist for ValuePlan and Opens<Vec<FieldValues>> for Plan, but no Opens<Vec<R>> for ValuePlan, and the blanket does not reach Vec<Encrypted<T>> (it implements DecryptInto<Vec<S>>, not DecryptInto<S>). Probe: cipher.encrypt(&ages).using(&age_plan).await? compiles; cipher.open(out).using(&age_plan).await is error[E0277]: OpenUsing<…> is not a future. The module doc's "a saved plan over a slice or a Vec settles every row in one" holds for the write only.

Nits

  1. packages/stack-encrypt/src/plan/chain.rs:259 — open is the right call (no overloading; StackCipher::decrypt(ct, aad) keeps its form; open is already the engine's word: target::open, Opening, "opens leaves"). What should move is the design doc on #1052, whose Rust API block and lowering table still say cipher.decrypt(row).using(&plan), plus a one-line glossary entry so the verb split is recorded. Also: the glossary lists row under Avoid for a field-by-field record, and this API uses row as its parameter name and doc noun throughout; pick one side.
  2. packages/stack-encrypt/src/plan/error.rs:94 — IndexNotDeclared prints "declares no match index" when the field does declare match, with other options (the Match::<Shingles> test). Render the options or add a variant.
  3. packages/stack-encrypt/src/plan/chain.rs:730 — "KeysetMismatch, before any request": resolve loads each named keyset (cipher.keyset(name), a ZeroKMS round trip) before it can compare ids. Say "before any key request", which is what the test asserts.
  4. packages/stack-encrypt/src/plan/error.rs:7 (and src/cipher.rs:223) — "raised before any key is requested" is not true of FieldValues::take, which returns NotInValue / FieldType on a record that has already been decrypted.
  5. packages/stack-encrypt/src/plan/values.rs:210 — a wrong-type take pushes the slot back at the end, so names() ("in order") changes after a failed take; insert(at, ..) keeps it stable.

Checked and found fine

Every build() rule from #1057 exists and fires in prepare, before any key request: duplicate field, non-plain context and segment, shared identity (default and pinned), passthrough on an indexed field in both orders, duplicate index (match with differing options counted as one kind), value/plan mismatch both ways (at build through Fields::schema(), at run through inspect zipped first so its error wins over pick's), unresolvable type. A query derives under exactly the write's label including .extend(parts): both sides go through DeclaredContext::under(label) → caller.extend(own) → (label)/parts, and a different extension does not match. all((..)) and slice/Vec runs are one generate_keys and one retrieve_keys per keyset via Pending::zip / Pending::all, counted by the tests. extend reaches every field because the one DeclaredContext is cloned into each zip arm. identity changes only the label segment; the name stays the record key and the Fields lookup. The byte-identity test compares generated descriptors, term bytes, and opens each spelling's ciphertext under the other's nonempty!(ctx).with(field) context, so descriptor, PRF context and AAD are all compared, not just a type. project_by and inspect do no cryptography (Pending::ready / failed only). Every FieldValues downcast is Result / Option; no unwrap, expect or unsafe in the diff; the Send + Sync + 'static on Lower / Opener are backed by the closures' own bounds. StackCipher had no encrypt, query or open before; KeysetCipher::encrypt(v, aad) and StackCipher::decrypt(ct, aad) are untouched. In all, one named keyset scoping every opening in the batch is forced by Pending::zip's check_scope regardless, so the scoped flag is honest. cargo test -p stack-encrypt --test plan: 22 passed. Both commits GPG-signed; no Linear ids in the diff, the commits or the PR body; the one derive .stderr change is path qualification only.

coderdan added a commit that referenced this pull request Oct 5, 2026
Decrypt through a plan is open, because StackCipher::decrypt already
exists with another signature. A plan has two starts and takes its
context from exactly one of three sources. The typed verb encrypt_into
and the picker are Rust-only. The derive emits the plan only after its
grammar is narrowed and the plan's widened, so the two stay one grammar.
EQL types are assembled per language from standard outputs, with no
registry and no target name in the data grammar.

The lowering table now names what #1068, #1069 and #1071 shipped.
@coderdan
coderdan added this pull request to stack #1075 October 5, 2026 05:11
coderdan added a commit that referenced this pull request Oct 5, 2026
…rces, encrypt_into and pickers

The derive is to emit the plan, so the plan must say everything the
narrowed derive says. This adds, Rust-only:

- Two starts: Plan::fields() and Plan::value::<S>(), each with an optional
  .context(c); Plan::context(c).fields()/.with(..) keep working. The
  FieldPlan iterator on a built plan is renamed field_plans(), since
  Plan::fields() is now the start.
- Exactly one context source: the plan, the call
  (cipher.encrypt(&v).context(c).using(&plan), and .context(c) on query
  and open), or a context field of the value (.context_field(field)),
  carried as a passthrough and checked on open through ExpectedContext.
  Two is TwoContextSources at build or at the call; none is NoContext when
  the plan runs, before any key request.
- encrypt_into: the field form lowers to the target's own EncryptFrom
  under <context>/<identity>, exactly as the derive composes a
  record-typed field; the one-value form takes a target or a tuple of
  them. A field is a target or data verbs, never both (TargetWithVerbs).
  A typed field answers the queries its target declares.
- Pickers: every verb takes a name (Field<F> lookup) or a name with an
  accessor; a plan of pickers needs no Fields impl and skips the run-time
  field-name check.

Also the #1071 review: a pinned identity rescues a field name that is not
a plain segment (tuple fields "0", renamed columns); a Vec written through
a one-value plan opens through it in one request; a query whose match
options differ says so (IndexOptions); a failed FieldValues::take keeps
the record's order; the over-strong "before any request" claims are
scoped; the parameter and doc noun row is now record.
@coderdan
coderdan force-pushed the feat/stack-encrypt-plan-builder branch from 4938466 to b819146 Compare October 5, 2026 05:24
coderdan added a commit that referenced this pull request Oct 5, 2026
…rces, encrypt_into and pickers

The derive is to emit the plan, so the plan must say everything the
narrowed derive says. This adds, Rust-only:

- Two starts: Plan::fields() and Plan::value::<S>(), each with an optional
  .context(c); Plan::context(c).fields()/.with(..) keep working. The
  FieldPlan iterator on a built plan is renamed field_plans(), since
  Plan::fields() is now the start.
- Exactly one context source: the plan, the call
  (cipher.encrypt(&v).context(c).using(&plan), and .context(c) on query
  and open), or a context field of the value (.context_field(field)),
  carried as a passthrough and checked on open through ExpectedContext.
  Two is TwoContextSources at build or at the call; none is NoContext when
  the plan runs, before any key request.
- encrypt_into: the field form lowers to the target's own EncryptFrom
  under <context>/<identity>, exactly as the derive composes a
  record-typed field; the one-value form takes a target or a tuple of
  them. A field is a target or data verbs, never both (TargetWithVerbs).
  A typed field answers the queries its target declares.
- Pickers: every verb takes a name (Field<F> lookup) or a name with an
  accessor; a plan of pickers needs no Fields impl and skips the run-time
  field-name check.

Also the #1071 review: a pinned identity rescues a field name that is not
a plain segment (tuple fields "0", renamed columns); a Vec written through
a one-value plan opens through it in one request; a query whose match
options differ says so (IndexOptions); a failed FieldValues::take keeps
the record's order; the over-strong "before any request" claims are
scoped; the parameter and doc noun row is now record.
@coderdan

coderdan commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main (014a04a) with no conflicts and no content change; commit hashes changed, nothing else.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-05T06:40:30.419772Z b819146 Draft marked ready
🔒 Security Review ⚠️ Failed 2026-10-05T06:34:41.745269Z b819146 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b819146296

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/stack-encrypt/src/plan/chain.rs Outdated
Comment thread packages/stack-encrypt/src/plan/chain.rs
Comment thread packages/stack-encrypt/src/plan/values.rs
@coderdan
coderdan force-pushed the feat/stack-encrypt-plan-builder branch from b819146 to e16b336 Compare October 5, 2026 07:13
coderdan added a commit that referenced this pull request Oct 5, 2026
…rces, encrypt_into and pickers

The derive is to emit the plan, so the plan must say everything the
narrowed derive says. This adds, Rust-only:

- Two starts: Plan::fields() and Plan::value::<S>(), each with an optional
  .context(c); Plan::context(c).fields()/.with(..) keep working. The
  FieldPlan iterator on a built plan is renamed field_plans(), since
  Plan::fields() is now the start.
- Exactly one context source: the plan, the call
  (cipher.encrypt(&v).context(c).using(&plan), and .context(c) on query
  and open), or a context field of the value (.context_field(field)),
  carried as a passthrough and checked on open through ExpectedContext.
  Two is TwoContextSources at build or at the call; none is NoContext when
  the plan runs, before any key request.
- encrypt_into: the field form lowers to the target's own EncryptFrom
  under <context>/<identity>, exactly as the derive composes a
  record-typed field; the one-value form takes a target or a tuple of
  them. A field is a target or data verbs, never both (TargetWithVerbs).
  A typed field answers the queries its target declares.
- Pickers: every verb takes a name (Field<F> lookup) or a name with an
  accessor; a plan of pickers needs no Fields impl and skips the run-time
  field-name check.

Also the #1071 review: a pinned identity rescues a field name that is not
a plain segment (tuple fields "0", renamed columns); a Vec written through
a one-value plan opens through it in one request; a query whose match
options differ says so (IndexOptions); a failed FieldValues::take keeps
the record's order; the over-strong "before any request" claims are
scoped; the parameter and doc noun row is now record.
@coderdan

coderdan commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto #1069's new head (9d10ee7, which folds TermKind into IndexSpec). No conflicts and no change to this PR's own two commits; stack-encrypt + derive nextest: 405 passed.

@coderdan
coderdan requested a review from auxesis October 5, 2026 07:16

@cipherstash-bot cipherstash-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommendation: 🟡 merge after changes

One change before merge: add a test that runs .extend(..) through a one-value plan (ValuePlan). Today, a change that ignores the caller's extension on that path still passes every test. The other findings can wait for a follow-up or are optional.

I ran every proposed test at this commit in a scratch copy of tests/plan.rs, and all passed. Then I restored the file. I also reproduced the two behaviour findings with scratch tests: all with an unnamed open, and passthrough field names.

This review does not repeat two issues, because they are already open as Codex threads:

  • all resolves named keysets through the first chain's cipher (chain.rs:781).
  • The chain resolves a named keyset before it validates the plan (chain.rs:172).

This review does not rate those two issues. Read the Codex threads to decide whether they must be fixed before merging. #1074 fixes the items in the author's own review.

Other findings not posted as comments

  • Optional: #[allow(non_snake_case)] at packages/stack-encrypt/src/plan/chain.rs:768 turns off a lint. The lint is needed because the all_of! macro uses the type parameter names (A, B, ...) as variable names. Pass separate lower-case identifiers for the bindings to the macro, and remove the allow.
How this review was made
Agent Model Review type Result
claude claude-opus-5-5 test-gap 3 found, 3 posted
claude claude-opus-5-5 rust 3 found, 3 posted
codex gpt-5.6-terra test-gap 1 found, 1 posted
codex gpt-5.6-terra rust 2 found, 0 posted

Synthesis: claude-opus-5-5 merged the findings, removed duplicates and dropped findings it could not confirm in the code. 0 posted finding(s) were raised by two or more models.

Plain language: claude-opus-5-5 read every comment as a new reader would. 6 comment(s) had a problem that stopped the reader acting; it rewrote 6. It also rewrote the review body.

Stack: position 3 of 6 (#1068, #1069, *️⃣ #1071, #1073, #1074, #1076). *️⃣ marks this pull request.

Context loaded: the description, 3 linked issue(s) and 24 discussion entries.

Comment thread packages/stack-encrypt/src/plan/build.rs
Comment thread packages/stack-encrypt/src/plan/chain.rs
Comment thread packages/stack-encrypt/src/plan/build.rs
Comment thread packages/stack-encrypt/src/plan/build.rs
Comment thread packages/stack-encrypt/src/plan/chain.rs Outdated
Comment thread packages/stack-encrypt/src/plan/chain.rs
Comment thread packages/stack-encrypt/src/plan/build.rs

@auxesis auxesis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this @coderdan.

Approving, and have added comments noting which review feedback must be addressed before merging.

@auxesis
auxesis force-pushed the feat/stack-encrypt-plan-builder branch from e16b336 to e511a47 Compare October 5, 2026 08:17
@coderdan

coderdan commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Rebased and review feedback addressed. New head: 3372419b2.

Rebase. The branch was already rebased onto the new base (7fe56ec79, feat/stack-encrypt-engine-plan-pieces) by a push at e511a47c6; its two commits are byte-identical to my own rebase of them, so the new commits sit on top of e511a47c6 and this push was a fast-forward. No conflicts.

New commits

  • 93963c53d fix: a failed FieldValues::take keeps the record's order (moved down from feat(stack-encrypt): plan grammar gains two starts, run-time and field contexts, encrypt_into and pickers #1074, which owns no FieldValues code)
  • c3aa3a69e fix: check a chain before loading the keyset it names (Operation::check; Runs::check / Opens::check)
  • c273ae5f0 fix: all() refuses chains from different ciphers (PlanError::MixedCiphers) and lets an unnamed open read any keyset
  • 8821f6682 fix: a passthrough field's name need not be a label segment (FieldPlan::label is Option)
  • 1c1a878ce test: plan_build fuzz target (fuzz:plan-build, wired into both fuzz.yml matrices and docs/fuzzing.md)
  • a44a66e33 feat: IntoLabel for &String, KeysetChoice: From<&String>
  • 51fdafe88 test: extend through a one-value plan, the index verb, a second extend
  • 3372419b2 docs: CHANGELOG ### Added bullets for the plan module (nothing renames an existing public name: Error is #[non_exhaustive] and StackCipher had no encrypt/query/open)

Threads. Every open thread got a reply with its commit and was resolved: the three Codex threads, the seven cipherstash-bot threads, and coderdan's seven nits. Six of those nits are fixed in #1074. The values.rs order nit is fixed here now.

Verification (at the final tree)

  • mise x --env test -- cargo nextest run -p stack-encrypt -p stack-encrypt-derive --all-features: 418 passed (trybuild ui included)
  • mise run test:doc, mise run doc: pass, no warnings
  • cargo fmt --all --check, cargo clippy --locked --no-deps --workspace --all-targets --all-features -- -D warnings, cargo clippy -p stack-encrypt --no-default-features -- -D warnings: clean
  • cargo check --tests -p stack-encrypt -p stack-encrypt-derive --all-features on every commit: clean
  • cargo mutants -p stack-encrypt --in-diff <whole PR diff vs base>: 252 mutants, 0 missed after the fix. The run first found 2 missed: the provided Runs::check / Opens::check defaults were equivalent to their own mutant. Their bodies are now plain Ok(()), which yields no mutant.
  • fuzz:plan-build: 60 s, 3.3 M runs, no crash; -runs=0 seed replay passes
  • the vitest wiring suites crates-ci, workflow-paths-filter-parity and cargo-lock-freshness: pass

coderdan added a commit that referenced this pull request Oct 5, 2026
…urces, encrypt_into and pickers

The derive is to emit the plan, so the plan must say everything the
narrowed derive says. This adds, Rust-only:

- Two starts: Plan::fields() and Plan::value::<S>(), each with an optional
  .context(c); Plan::context(c).fields()/.with(..) keep working. The
  FieldPlan iterator on a built plan is renamed field_plans(), since
  Plan::fields() is now the start.
- Exactly one context source: the plan, the call
  (cipher.encrypt(&v).context(c).using(&plan), and .context(c) on query
  and open), or a context field of the value (.context_field(field)),
  carried as a passthrough and checked on open through ExpectedContext.
  Two is TwoContextSources at build or at the call; none is NoContext when
  the plan runs, before any key request.
- encrypt_into: the field form lowers to the target's own EncryptFrom
  under <context>/<identity>, exactly as the derive composes a
  record-typed field; the one-value form takes a target or a tuple of
  them. A field is a target or data verbs, never both (TargetWithVerbs).
  A typed field answers the queries its target declares.
- Pickers: every verb takes a name (Field<F> lookup) or a name with an
  accessor; a plan of pickers needs no Fields impl and skips the run-time
  field-name check.

Also the #1071 review: a pinned identity rescues a field name that is not
a plain segment (tuple fields "0", renamed columns); a Vec written through
a one-value plan opens through it in one request; a query whose match
options differ says so (IndexOptions); the over-strong "before any
request" claims are scoped; the parameter and doc noun row is now record.

Runs::check and Opens::check take the context the call names, so a
chain refuses a missing or doubled context, and a context field's
mismatch, before it loads a keyset.

The plan_build fuzz target follows: it builds plans with or without a
context and with context fields, and asserts that a context given twice
is refused as TwoContextSources and that a contextless plan keys each
sealed field under one plain identity segment.

BREAKING CHANGE: the unreleased plan API changes shape (nothing here is
in 0.2.0). Plan::fields() starts a fields plan, so a built plan's field
iterator is Plan::field_plans(). A one-value plan is
ValuePlan<S, Indexed<X>> or ValuePlan<S, Typed<T>> in place of
ValuePlan<S, X>. Plan::label and ValuePlan::label return Option<&Label>.
Runs::pending, Runs::check, Opens::decryption, Opens::check,
Plan::encryption, Plan::decryption and ValuePlan::encryption take the
context the call names. The field verbs take a FieldRef (a name still
works).
Two front ends drove the engine by hand (the derive and dynamic::record)
and a query restated its field's context by hand, so the three could
drift and a drifted query matched nothing, silently. A plan is the saved
tail of an encrypt call with the value left out; the write, the query
and the read all take it, so none of them can respell a context.

cipher.encrypt(&v) starts the chain: .context(c) seals one tree,
.with(indexes) indexes one value, and .fields() seals each top-level
field under <context>/<field> with encrypt / encrypt_index / index /
passthrough. .keyset(..) and .extend(parts) go anywhere, and .await
finishes it. Plan::context(..) is the same chain without a value;
build() validates the whole plan once. cipher.query(v).using(&field_plan)
derives a term under the write's label and refuses an undeclared index;
cipher.open(row).using(&plan) decrypts what can come back. all((a, b))
settles several chains in one ZeroKMS request.

Every call lowers to an existing combinator (indexed, ciphertext,
Indexes::operations, passthrough, under, zip, KeysetCipher::run, open,
run_decryption). The one engine addition is a crate-internal by-name
projection and a source check, so a fields plan can pick a field out of
a value it only knows at run time. No new cryptographic operation.

The decrypt entry is named open: StackCipher::decrypt is the
cipher-directed decrypt and keeps its two-argument form.
… rule

cargo-mutants found that dropping a chain's named keyset, or widening the
passthrough-and-indexed check, went unnoticed. Each chain kind now asserts
the keyset its keys and terms come from, and an indexed or passthrough
field named twice without the other is a duplicate field.
A take at the wrong type removed the field and pushed it back at the end,
so names(), documented as in order, changed after a failed read. The slot
now goes back at the position it came from.

This fix first landed higher in the stack, in the plan grammar PR; it
moves here because this PR owns FieldValues.
Awaiting a chain with `.keyset(name)` loaded the keyset (a ZeroKMS lookup
and an index-key load) before `prepare` validated the plan, so an invalid
context, a duplicate index, a value or stored record that does not match
the plan, or a query of an undeclared index made a network request first,
and a failed lookup could hide the plan's own refusal.

`Operation` gains `check`: the plan, and the value or record it runs over,
checked with no I/O and no keyset. A single chain and every chain in
`all((..))` are checked before any keyset is resolved. `Runs` and `Opens`
gain a provided `check` for the value and record checks; a fields plan
answers it with what running it checks (the field names both ways and each
field's type, through a per-field probe) and what opening checks (names
both ways and the stored type each opening reads). Plan decryption now
runs the same record check, so the two cannot disagree.
…ead any keyset

Two problems with batching chains in `all((..))`:

- It resolved every chain's named keyset on the first chain's cipher, so
  `all((a.encrypt(..), b.encrypt(..).keyset("tenant")))` could run b's
  value under a's "tenant". A batch settles through one client, so chains
  started on different ciphers are now refused with the new
  `PlanError::MixedCiphers`, before any keyset is loaded (pointer identity:
  `StackCipher` is not `Clone`).
- An `open` chain that named no keyset was zipped into the batch and
  confined to the batch keyset, so a row that opens alone failed with
  `ForeignKeyset` beside a write. Such an opening now settles in its own
  group beside the batch (one more retrieve per keyset it reads), so it
  means in a batch what it means alone. A failed chain always lands in the
  batch group, which is settled first, so a refusal still makes no request.

`prepare`'s `scoped` flag is now per chain (whether this chain named the
keyset), and the batch's rule for a chain naming none is written down on
`KeysetChoice` and `all`. `Operation::Output` is bounded by `MaybeSend`,
which the batch's routed pendings need across an await.

Also drops the `#[allow(non_snake_case)]` on the batch macro: the
bindings are lower-case identifiers of their own.
…gment

`build()` checked every field's name as a label segment, but a passthrough
field is under no label: its lowering and opening ignore the context, and
its name is only the record key. So a record with a passthrough column
named `2fa_enabled` or `created(utc)` could not get a plan, which a
binding mapping arbitrary keys onto the builder would hit, and a
passthrough `id` was refused as sharing an identity with a sealed field
pinned to `id`.

A passthrough field now has no label (`FieldPlan::label` is
`Option<&Label>`, `None` for one) and takes no part in the shared-identity
rule; a query of it is `IndexNotDeclared`, as before. A sealed or indexed
field's name and identity are checked as before.
A binding builds plans from names its caller supplies, so
`FieldsBuilder::build` parses untrusted text as much as the byte decoders
do. The new `plan_build` target drives it with `Arbitrary`-derived inputs
(context, field names, pinned identities and verbs, from a small colliding
alphabet or free text) and checks three things: building and rendering
never panic; every sealed or indexed field of a plan that builds has the
label `<context>/<identity>`, which renders and parses back to itself; and
a plan of distinct passthrough names under a plain context always builds,
whatever the names are.

Wired as `fuzz:plan-build` in the crate's tasks, in both matrices of
fuzz.yml (the per-PR seed replay and the nightly campaign) and in
docs/fuzzing.md, with four committed seeds (two plans that build, two
refused). A 60 s local campaign (3.3 M runs) found nothing.
`IntoLabel` had impls for `&str`, `String`, `Label` and `&Label`, and Rust
does not deref-coerce into a generic `impl IntoLabel` parameter, so
`Plan::context(&name)` and `.context(&name)` with `name: String` did not
compile, and `.keyset(&tenant_name)` neither. A context or keyset name
held in a config `String` had to be written `.as_str()` or cloned.

Adds `IntoLabel for &String` and `From<&String> for KeysetChoice` (the
same value the `&str` impl builds), with a test calling each.
…erb and a second extend

Three paths no test held:

- `.extend(..)` through a `ValuePlan`: the write asks for its data key
  under `(users/age)/7u64`, a query term matches only under the same
  extension, and the value opens only with it (no extension and another
  extension are both `Aead`). Passing a default context instead of the
  caller's on that path now fails a test.
- The `index` verb: an index-only field asks for no data key, the query of
  the field matches its stored term, and the terms are byte-identical to
  what `encrypt_index` writes for the same field.
- A second `.extend(..)` replaces the first: every data key is asked for
  under the last extension only, and the record opens with it alone.
The plan module is new public API: the chain on the cipher, saved fields
and one-value plans, `all`, `PlanError` (in the new `Error::Plan`), and the
fixes from review (a passthrough name under no label, `&String` contexts
and keyset names, a chain checked before its keyset loads). `Error` is
`#[non_exhaustive]` and `StackCipher` had no `encrypt`, `query` or `open`
before, so nothing here changes an existing public name.
@coderdan
coderdan force-pushed the feat/stack-encrypt-plan-builder branch from 3372419 to 587be2e Compare October 5, 2026 16:35
@coderdan
coderdan merged commit 327b4db into main Oct 5, 2026
48 checks passed
@coderdan
coderdan deleted the feat/stack-encrypt-plan-builder branch October 5, 2026 17:22
coderdan added a commit that referenced this pull request Oct 5, 2026
…urces, encrypt_into and pickers

The derive is to emit the plan, so the plan must say everything the
narrowed derive says. This adds, Rust-only:

- Two starts: Plan::fields() and Plan::value::<S>(), each with an optional
  .context(c); Plan::context(c).fields()/.with(..) keep working. The
  FieldPlan iterator on a built plan is renamed field_plans(), since
  Plan::fields() is now the start.
- Exactly one context source: the plan, the call
  (cipher.encrypt(&v).context(c).using(&plan), and .context(c) on query
  and open), or a context field of the value (.context_field(field)),
  carried as a passthrough and checked on open through ExpectedContext.
  Two is TwoContextSources at build or at the call; none is NoContext when
  the plan runs, before any key request.
- encrypt_into: the field form lowers to the target's own EncryptFrom
  under <context>/<identity>, exactly as the derive composes a
  record-typed field; the one-value form takes a target or a tuple of
  them. A field is a target or data verbs, never both (TargetWithVerbs).
  A typed field answers the queries its target declares.
- Pickers: every verb takes a name (Field<F> lookup) or a name with an
  accessor; a plan of pickers needs no Fields impl and skips the run-time
  field-name check.

Also the #1071 review: a pinned identity rescues a field name that is not
a plain segment (tuple fields "0", renamed columns); a Vec written through
a one-value plan opens through it in one request; a query whose match
options differ says so (IndexOptions); the over-strong "before any
request" claims are scoped; the parameter and doc noun row is now record.

Runs::check and Opens::check take the context the call names, so a
chain refuses a missing or doubled context, and a context field's
mismatch, before it loads a keyset.

The plan_build fuzz target follows: it builds plans with or without a
context and with context fields, and asserts that a context given twice
is refused as TwoContextSources and that a contextless plan keys each
sealed field under one plain identity segment.

BREAKING CHANGE: the unreleased plan API changes shape (nothing here is
in 0.2.0). Plan::fields() starts a fields plan, so a built plan's field
iterator is Plan::field_plans(). A one-value plan is
ValuePlan<S, Indexed<X>> or ValuePlan<S, Typed<T>> in place of
ValuePlan<S, X>. Plan::label and ValuePlan::label return Option<&Label>.
Runs::pending, Runs::check, Opens::decryption, Opens::check,
Plan::encryption, Plan::decryption and ValuePlan::encryption take the
context the call names. The field verbs take a FieldRef (a name still
works).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stack-encrypt: one chained plan builder as the single front door, so writes, queries and reads cannot drift

3 participants