stack-encrypt: a plan cannot take its context from the caller or a field, cannot lay a field out as another type, and silently drops match options on the wire

[coderdan]

Background

stack-encrypt (packages/stack-encrypt) is our Rust library for field-level encryption: each value is sealed under its own data key from ZeroKMS, our key management service, and can carry search terms ("indexes": equality, match, ORE, OPE) beside the ciphertext so an encrypted column can still be queried. Every sealed value is bound to a context, a label such as users/email that ties the data to where it belongs and that ZeroKMS sees as a descriptor.

A plan (the plan builder, #1057, built in #1071) is a saved, checked description of how a value is encrypted: its context and, field by field, what each field becomes (a ciphertext, search terms, or a value carried through unencrypted). The plan is meant to be the single front door: #1058 makes #[derive(EncryptFrom)] emit a plan, and #1062 makes EQL domains (the column shapes of EQL, our PostgreSQL encryption library) into plan field targets, including from a generated Go package. For either to work, a plan has to be able to say everything the derive says.

Problem

Mechanism. Even after the derive is narrowed to what a plan could reasonably express (#1078), the plan as built in #1071 is missing these pieces. Line numbers are against the #1073 branch, which the work stacks on:

1. A plan must be given its context when it is built. The only start is Plan::context(c) (src/plan/build.rs:200). The derive's plaintext = T records and its generic form take their context from the caller, at run time. A plan has no way to say "the caller supplies the context".
2. A plan cannot take its context from a field of the value. The derive's #[stash(context_field)], which EQL's Rust types use, seals every other field under the value of one field (for example a tenant id) and checks it on decrypt. A plan has no equivalent.
3. A plan field cannot be laid out as another target type. The derive composes a field whose type is itself a derived record, and a plaintext = String record is a tuple of targets over one value. A plan can only use its own verbs (encrypt, encrypt_index, index, passthrough), so it cannot express either, nor an EQL domain as a field target (Go and Rust plans cannot target EQL domains, so Go writes rows EQL cannot query — make domains plan field targets #1062).
4. A plan reads a field only by name, through the value's Field<F> impl (src/plan/build.rs:388). Code that generates a plan (the derive, or a Go generator) would need every value type to implement those traits, and needs a turbofish per field. There is no way to hand the plan an accessor.
5. A field name that is not a plain label segment cannot be rescued by a pinned identity. A tuple struct's field 0 cannot be keyed as value, which the derive needs to lower tuple-struct records.
6. Match options have no wire form, and are dropped silently. When a plan is written out as data (for the Go binding or a saved plan), a match index with non-default options (a different tokenizer or filter size) is written as plain "match" (src/target/index.rs:91), and reading "match" back always yields the default options (src/dynamic/term.rs:80). The query side then derives terms under different options than the stored rows, so every query returns zero rows with no error. This was found in review of feat(stack-encrypt): passthrough, Index/Indexes, Encrypted<Terms> and field types for the plan builder #1069.

Impact. #1058 cannot emit a plan for every derive shape with identical bytes, and #1062 cannot make domains plan field targets, until these exist. Item 6 is a silent data-correctness bug: wrong search results, no error.

Why nothing catches it today. No test compares a derive's output with an equivalent plan, and nothing round-trips an index through its data form.

Proposal

1. Two starts, each with an optional .context(c): Plan::fields() for a field-by-field plan, and Plan::value::<S>() for a one-value plan. Plan::context(c) keeps working.
2. Exactly one context source, from one of three places: the plan (.context(c) when built), the call that runs it (.context(c) on encrypt, query and open), or a field of the value (.context_field(field), mirroring the derive's context_field, with the same check on open before any key is requested). Two sources is an error at build or at the call; none is an error when the plan runs, before any key request.
3. encrypt_into::<T>: lay a field (or a one-value plan) out as target type T, through T's own EncryptFrom, under <context>/<field>. A tuple of targets is itself a target. A field is either verbs or one target, never both. Queries on a typed field answer only the indexes the target names (a defaulted EncryptFrom::indexes()); anything else is refused, never derived.
4. Pickers: every field verb also takes a name plus an accessor, pick("email", |u: &User| &u.email), which needs no Field<F> impl.
5. A pinned identity rescues a field name that is not a plain segment: only the identity enters the label.
6. Match options on the wire: an explicit conversion from an index to its data form that round-trips every kind and refuses a match index with non-default options, rather than dropping them. A loud refusal beats a silent zero-row miss. The "json" key stays reserved for the JSON index (stack-encrypt has no searchable JSON — add a Json index that seals a document's entries under one data key #1060).
7. Proof: tests that show, byte for byte, that a plan using each addition writes what the corresponding derive writes (same ZeroKMS descriptors, same term bytes, each record opens through the other).

Verified against the code (on the #1073 branch): items 1, 2, 4 and 6 of the Problem, at the cited lines. From the fix's PR: item 5, and that the Go binding's data form of a plan is otherwise unchanged.

Relationship to other work

• Blocked by stack-encrypt-derive: #[derive(EncryptFrom)] accepts five shapes a plan cannot express, so it can never emit the plan with identical bytes #1078 (the derive narrowing): the plan grows to meet the narrowed derive, not the old one.
• Blocks stack-encrypt-derive: #[derive(EncryptFrom)] hand-writes combinators — emit the plan builder instead #1058 (the derive emits the plan) and Go and Rust plans cannot target EQL domains, so Go writes rows EQL cannot query — make domains plan field targets #1062 (EQL domains as plan field targets).
• Resolved by feat(stack-encrypt): plan grammar gains two starts, run-time and field contexts, encrypt_into and pickers #1074.