From e7eeecdd3ddd80d8208f34739a25d110e95e005e Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 01:00:41 -0700 Subject: [PATCH 01/14] feat(stack-encrypt)!: dynamic::record lowers into the plan builder Closes the second executor ADR-0007 names. dynamic::record used to walk a data plan itself: it called the term functions and the seal path per field, collected the pendings and merged them with Pending::all, and never touched the Encryption descriptions the derive and a Rust chain run. The rules therefore existed twice, and had drifted. It is now a lowering: the parsed plan is built into the same Plan::context(c).fields() a Rust chain writes, with encrypt, encrypt_index, index or passthrough per field, and run through KeysetCipher::run and the plan's opener. The module's own per-field loop, batching and output shaping are gone; what remains around the engine is an adapter from the wire value to FieldValues and back, and the fail-closed checks a binding needs before it has a cipher. A plan yields a Pending synchronously, so encrypt and decrypt lose their async: each checks and converts its input with no cipher and returns the plan's Pending, whose failure is the crate's Error. The guest awaits it in the same block_on it always had, and classifies Error::Plan as the caller's input (STATUS_ENCODING), which is what it is. Three decisions the ADR left to the implementation, and why: Context. A fields plan has one context and seals every field under /; the data grammar gave each field its whole context. A field's "context" must now be the field's label (a list of at least two plain segments, what a Go label= tag sends), optionally extended by scalar parts nested to the left as Go's ExtendContext nests them; the last segment is the identity, the rest the plan's context, and every field of a plan must share the prefix and the extension, which becomes the chain's .extend(..). A bare text part, an integer or a one-segment label is refused: the plan cannot express a field outside the record context, and the derive lost that in #1073 for the same reason. The grammar on the wire is otherwise unchanged, and so is the stored record shape. Leaf encoding. A derive over a bare u32 sealed four untagged bytes while a data plan sealed vitaminc's tagged FfiValue leaf, and the old rustdoc recorded that as by design. The declared "type" is the data form of the chain's ::: a field typed uint32 or string lowers to a u32 or String field and runs exactly the operations encrypt_index:: runs, so the derive and a data plan interchange ciphertexts and terms (pinned by a test that opens each with the other). Every other kind, and an untyped field, seals the tagged encoding, because vitaminc gives those kinds no bare Rust leaf (u64, bool and the floats have no Encrypt) and an untyped field has no type to name; that half is transitional until "type" is required (#1082). What the builder lacked, added once for every author rather than in the record module: Vec as an index set sized at run time (an empty one is PlanError::EmptyIndexes when it runs); IndexSpec as an Index of u32, of String and of the new dynamic::Value, each with TermBytes as its term, so an index named as data runs through indexed() like one named as a type; Value, an FfiValue as a plan field's plaintext, Clone by deep copy into fresh Protected payloads, since the borrowed engine clones what it consumes and FfiValue deliberately is not Clone; DeclaredContext::with, so an extension of several parts nests to the left as NonEmpty::with does; and a crate-internal chosen() constructor, the one description whose operation is picked by the value it is handed, which is the one step that stays dynamic. dynamic::term runs the same dispatch. Output::Passthrough joins the wire grammar ("passthrough", a field's only output) so a record can carry a field whole, as the ADR's amendment says generated Go code will. Two sealed fields under one label are now refused (the builder's SharedIdentity); the executor accepted them. The guest's native tests spell their plan contexts as labels; the fuzz target for check_record generates label-shaped contexts so plans still parse. The Go module's API and exports are unchanged; only the Rust behind se_encrypt_record and se_decrypt_record changed. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- .../golang/stackencrypt/guest/src/ops.rs | 29 +- .../golang/stackencrypt/guest/src/status.rs | 15 + .../stackencrypt/guest/tests/native_ops.rs | 77 +- packages/stack-encrypt/CHANGELOG.md | 51 + .../fuzz/fuzz_targets/check_record.rs | 14 +- packages/stack-encrypt/src/dynamic/mod.rs | 66 +- packages/stack-encrypt/src/dynamic/record.rs | 2396 ++++++++++++----- packages/stack-encrypt/src/dynamic/term.rs | 291 +- packages/stack-encrypt/src/dynamic/value.rs | 199 ++ packages/stack-encrypt/src/plan/error.rs | 6 + packages/stack-encrypt/src/plan/mod.rs | 10 + packages/stack-encrypt/src/target/context.rs | 47 +- packages/stack-encrypt/src/target/index.rs | 33 + packages/stack-encrypt/src/target/mod.rs | 2 + .../stack-encrypt/src/target/operations.rs | 26 + 15 files changed, 2366 insertions(+), 896 deletions(-) create mode 100644 packages/stack-encrypt/src/dynamic/value.rs diff --git a/languages/golang/stackencrypt/guest/src/ops.rs b/languages/golang/stackencrypt/guest/src/ops.rs index c294d796a..e0634a6f3 100644 --- a/languages/golang/stackencrypt/guest/src/ops.rs +++ b/languages/golang/stackencrypt/guest/src/ops.rs @@ -169,7 +169,7 @@ pub async fn term( kind: u32, ) -> Result, u32> where - K: DataKeySource + Sync, + K: DataKeySource + Sync + 'static, { // The same proof every stack-encrypt leaf demands: an empty context is // `STATUS_ENCODING` here, before any derivation. @@ -212,11 +212,14 @@ fn parse_term(value: FfiValue, kind: u32) -> Result<(Scalar, IndexSpec), u32> { /// /// Both arguments are codec-encoded: the plan is the object /// [`dynamic::record::plan`] parses, the source an object of -/// `{ field: scalar }` (one record) or an array of them (a batch). The +/// `{ field: value }` (one record) or an array of them (a batch). The /// result is a codec-encoded ciphertext tree — per record a map of /// `field → { output-key → node }`. /// -/// All rows and fields seal in one batched `generate_keys`; that batch +/// The plan is lowered into the engine's plan builder and run there +/// (ADR-0007): [`dynamic::record::encrypt`] hands back the plan's `Pending` +/// with every key request queued and nothing sent, and awaiting it here is +/// the one batched `generate_keys` for all rows and fields. That batch /// reaches ZeroKMS as one request per /// [`ClientOpts::max_keys_per_req`](stack_kms::ClientOpts::with_max_keys_per_req) /// keyed leaves (500 by default, sent sequentially: the guest pins @@ -230,22 +233,25 @@ pub async fn encrypt_record( plan: &[u8], ) -> Result, u32> where - K: DataKeySource + Sync, + K: DataKeySource + Sync + 'static, { let plan = dynamic::record::plan(decode_value(plan)?).map_err(|e| status_for_dynamic(&e))?; let tree = dynamic::record::encrypt(cipher, decode_value(source)?, &plan) + .map_err(|e| status_for_dynamic(&e))? .await - .map_err(|e| status_for_dynamic(&e))?; + .map_err(|e| status_for_error(&e))?; encode_tree(tree) } /// Decrypt a record — or a batch — produced by [`encrypt_record`] under the -/// same plan. Only the `"c"` outputs participate (terms are one-way). +/// same plan. Only the `"c"` and `"passthrough"` outputs participate (terms +/// are one-way). /// -/// One batched `retrieve_keys` per invocation, dispatched as one ZeroKMS -/// call per 500 keyed leaves and, under [`Scope::Client`], per keyset the -/// leaves were sealed under. The output buffer contains plaintext — the ABI -/// layer's ownership rules govern its wiping. +/// The plan's opener runs in the engine; awaiting it here is the one +/// batched `retrieve_keys` per invocation, dispatched as one ZeroKMS call +/// per 500 keyed leaves and, under [`Scope::Client`], per keyset the leaves +/// were sealed under. The output buffer contains plaintext — the ABI layer's +/// ownership rules govern its wiping. pub async fn decrypt_record( scope: Scope<'_, K>, record: &[u8], @@ -256,8 +262,9 @@ where { let plan = dynamic::record::plan(decode_value(plan)?).map_err(|e| status_for_dynamic(&e))?; let value = dynamic::record::decrypt(scope, decode_tree(record)?, &plan) + .map_err(|e| status_for_dynamic(&e))? .await - .map_err(|e| status_for_dynamic(&e))?; + .map_err(|e| status_for_error(&e))?; encode_value(value) } diff --git a/languages/golang/stackencrypt/guest/src/status.rs b/languages/golang/stackencrypt/guest/src/status.rs index b985c0434..4647f1634 100644 --- a/languages/golang/stackencrypt/guest/src/status.rs +++ b/languages/golang/stackencrypt/guest/src/status.rs @@ -75,6 +75,11 @@ pub fn status_for_error(error: &stack_encrypt::Error) -> u32 { // A context that renders past ZeroKMS's descriptor limit is the // caller's input, refused before any request is sent. stack_encrypt::Error::DescriptorTooLong { .. } => STATUS_ENCODING, + // A plan refusal — a value or stored record that does not fit the + // plan it is run with, a typed field opened to another kind — is a + // statement about the caller's input, raised by the engine the + // record exports run their plans through. + stack_encrypt::Error::Plan(_) => STATUS_ENCODING, _ => STATUS_INTERNAL, } } @@ -296,6 +301,16 @@ mod tests { ); } + #[test] + fn a_plan_refusal_is_the_callers_input() { + assert_eq!( + status_for_error(&stack_encrypt::Error::Plan( + stack_encrypt::PlanError::NoContext + )), + STATUS_ENCODING + ); + } + #[test] fn a_foreign_keyset_is_its_own_status_and_scope_bugs_are_internal() { let (a, b) = (uuid::Uuid::from_u128(1), uuid::Uuid::from_u128(2)); diff --git a/languages/golang/stackencrypt/guest/tests/native_ops.rs b/languages/golang/stackencrypt/guest/tests/native_ops.rs index 541f548c6..86561438e 100644 --- a/languages/golang/stackencrypt/guest/tests/native_ops.rs +++ b/languages/golang/stackencrypt/guest/tests/native_ops.rs @@ -145,6 +145,12 @@ fn s(value: &str) -> FfiValue { FfiValue::String(value.into()) } +/// The label `users/` as a plan field spells its context: a list of +/// plain segments, what a Go `label=users/` tag sends. +fn label(field: &str) -> FfiValue { + FfiValue::Array(vec![s("users"), s(field)]) +} + /// The plan shape the record tests share — an ORE-indexed integer and a /// match-indexed string, both stored — under whatever context `ctx` gives /// each field. Output order is fixed here, and the tests index into it. @@ -167,9 +173,9 @@ fn plan_under(ctx: impl Fn(&str) -> FfiValue) -> Vec { ])) } -/// The plan used by the record tests: `plan_under` with flat contexts. +/// The plan used by the record tests: `plan_under` with each field's label. fn plan() -> Vec { - plan_under(|field| s(&format!("users/{field}"))) + plan_under(label) } /// A one-field plan storing only the ciphertext, under `context`. @@ -711,7 +717,7 @@ fn a_passthrough_source_value_is_refused_a_ciphertext_slot() { let plan = encode(obj(vec![( "age", obj(vec![ - ("context", s("users/age")), + ("context", label("age")), ("outputs", FfiValue::Array(vec![s("c")])), ]), )])); @@ -759,14 +765,14 @@ fn record_terms_equal_the_native_derivations_and_probe_them() { let eq_probe = block_on( cipher .default_keyset() - .equality_term(34u32, nonempty!("users/age")), + .equality_term(34u32, nonempty!("users").with("age")), ) .expect("probe"); assert_eq!(term_bytes(&age_outputs[1].1), eq_probe.as_bytes()); let ore_probe = block_on( cipher .default_keyset() - .ore_term(34u32, nonempty!("users/age")), + .ore_term(34u32, nonempty!("users").with("age")), ) .expect("probe"); assert_eq!(term_bytes(&age_outputs[2].1), ore_probe.as_ref()); @@ -777,19 +783,20 @@ fn record_terms_equal_the_native_derivations_and_probe_them() { let match_probe = block_on( cipher .default_keyset() - .match_terms::("alice smith", nonempty!("users/name")), + .match_terms::("alice smith", nonempty!("users").with("name")), ) .expect("probe"); assert_eq!(term_bytes(&name_outputs[1].1), match_probe.to_bytes()); // And the "c" node is an ordinary value-model ciphertext bound to the - // field's context. + // field's label. let CipherText::Single(leaf) = &age_outputs[0].1 else { panic!("expected a single leaf for a scalar field"); }; let leaf = SealedValue::from_bytes(leaf).expect("frozen leaf"); - let value: FfiValue = block_on(cipher.decrypt(CipherText::Single(leaf), "users/age")) - .expect("native decrypt of a record field"); + let value: FfiValue = + block_on(cipher.decrypt(CipherText::Single(leaf), nonempty!("users").with("age"))) + .expect("native decrypt of a record field"); assert!(matches!(value, FfiValue::UInt32(34))); } @@ -797,11 +804,11 @@ fn record_terms_equal_the_native_derivations_and_probe_them() { // Structured contexts // ============================================================================= -/// The caller extension a Rust row gets from -/// `encrypt_into_with_context(row, 7u64)`: every field's context becomes -/// `("users/", 7u64)`. A plan spells it as a list. +/// The caller extension a Rust row gets from the chain's `.extend(7u64)`: +/// every field's label becomes `(users/)/7u64`. A plan spells it as a +/// list, the label nested inside. fn extended(field: &str) -> FfiValue { - FfiValue::Array(vec![s(&format!("users/{field}")), FfiValue::UInt64(7)]) + FfiValue::Array(vec![label(field), FfiValue::UInt64(7)]) } /// `plan()` under the extension. @@ -811,9 +818,9 @@ fn extended_plan() -> Vec { /// A plan whose context is a list seals exactly what the Rust derive seals /// under a caller-extended context: the stored terms are the native probes -/// under `nonempty!("users/age").with(7u64)`, the guest's own probe under -/// the list is the same bytes, and the `"c"` leaf opens natively under the -/// tuple. The flat context is a different domain, as it must be. +/// under `nonempty!("users").with("age").with(7u64)`, the guest's own probe +/// under the list is the same bytes, and the `"c"` leaf opens natively under +/// the tuple. The flat label is a different domain, as it must be. #[test] fn a_structured_plan_context_seals_what_the_native_extended_context_does() { let cipher = cipher(); @@ -831,7 +838,7 @@ fn a_structured_plan_context_seals_what_the_native_extended_context_does() { panic!("expected an output map for the first field"); }; - let native = nonempty!("users/age").with(7u64); + let native = nonempty!("users").with("age").with(7u64); let eq_probe = block_on(cipher.default_keyset().equality_term(34u32, native)).expect("probe"); assert_eq!(term_bytes(&age_outputs[1].1), eq_probe.as_bytes()); let ore_probe = block_on(cipher.default_keyset().ore_term(34u32, native)).expect("probe"); @@ -839,7 +846,7 @@ fn a_structured_plan_context_seals_what_the_native_extended_context_does() { let flat_probe = block_on( cipher .default_keyset() - .equality_term(34u32, nonempty!("users/age")), + .equality_term(34u32, nonempty!("users").with("age")), ) .expect("probe"); assert_ne!( @@ -863,7 +870,7 @@ fn a_structured_plan_context_seals_what_the_native_extended_context_does() { let match_probe = block_on( cipher .default_keyset() - .match_terms::("alice smith", nonempty!("users/name").with(7u64)), + .match_terms::("alice smith", nonempty!("users").with("name").with(7u64)), ) .expect("probe"); assert_eq!(term_bytes(&name_outputs[1].1), match_probe.to_bytes()); @@ -884,7 +891,7 @@ fn a_structured_plan_context_seals_what_the_native_extended_context_does() { fn a_natively_sealed_field_under_an_extended_context_opens_through_a_plan() { let cipher = cipher(); let keyset = cipher.default_keyset(); - let native = nonempty!("users/age").with(7u64); + let native = nonempty!("users").with("age").with(7u64); let sealed = block_on( FfiValue::UInt32(34) .encrypt_with_aad(&keyset, native) @@ -921,15 +928,16 @@ fn a_natively_sealed_field_under_an_extended_context_opens_through_a_plan() { block_on(ops::decrypt_record( Scope::Client(&cipher), &record, - &plan_with(s("users/age")) + &plan_with(label("age")) )), Err(STATUS_AUTH), - "the flat context is not the one it was sealed under" + "the flat label is not the one it was sealed under" ); } /// A plan context that is not a context — the wrong value kind, or empty -/// by the tuple rule — is refused at parse, before anything is sealed. +/// by the tuple rule — or that is a context but not a field's label is +/// refused at parse, before anything is sealed. #[test] fn a_structured_plan_context_is_validated_at_parse() { let cipher = cipher(); @@ -944,8 +952,11 @@ fn a_structured_plan_context_is_validated_at_parse() { ("a list of one empty string", FfiValue::Array(vec![s("")])), ( "a list with a float in it", - FfiValue::Array(vec![s("users/age"), FfiValue::Float64(7.0)]), + FfiValue::Array(vec![label("age"), FfiValue::Float64(7.0)]), ), + ("one text part, not a label", s("users/age")), + ("a one-segment label", FfiValue::Array(vec![s("users")])), + ("an integer", FfiValue::UInt64(7)), ] { assert_eq!( block_on(ops::encrypt_record( @@ -963,17 +974,18 @@ fn a_structured_plan_context_is_validated_at_parse() { "nothing seals under a context that is not one" ); - // Non-empty by the tuple rule: one part carries bytes. + // An extension part may be empty: the label carries the context, as a + // Go `Context.With("")` extends one. let sealed = block_on(ops::encrypt_record( &cipher.default_keyset(), &source, - &plan_with(FfiValue::Array(vec![s(""), FfiValue::UInt64(7)])), + &plan_with(FfiValue::Array(vec![label("f"), s("")])), )) - .expect("an integer part is never empty"); + .expect("a label extended by an empty part is never empty"); assert!(block_on(ops::decrypt_record( Scope::Client(&cipher), &sealed, - &plan_with(FfiValue::Array(vec![s(""), FfiValue::UInt64(7)])) + &plan_with(FfiValue::Array(vec![label("f"), s("")])) )) .is_ok()); } @@ -1102,12 +1114,13 @@ fn an_empty_plan_context_is_refused_before_anything_is_sealed() { Err(STATUS_ENCODING) ); - // A context of unusual bytes is still a context: it seals, and opens. - let odd = String::from_utf8(vec![0u8; 8]).expect("nul bytes are valid utf-8"); + // A label of unusual but plain text is still a label: it seals, and + // opens. (Control characters are not plain, so a NUL is refused at + // parse as any non-label is.) let odd_plan = encode(obj(vec![( "f", obj(vec![ - ("context", s(&odd)), + ("context", FfiValue::Array(vec![s("naïve users"), s("f")])), ("outputs", FfiValue::Array(vec![s("c")])), ]), )])); @@ -1403,7 +1416,7 @@ fn record_validation_refuses_what_encrypt_record_refuses() { // A passthrough under a ciphertext output, with no term output to mask // it (the invariant `a_passthrough_source_value_is_refused_a_ciphertext_slot` pins). - let ct_only = single_field_plan("age", s("users/age")); + let ct_only = single_field_plan("age", label("age")); let passthrough = encode(obj(vec![( "age", FfiValue::Passthrough(Box::new(FfiValue::UInt32(29))), diff --git a/packages/stack-encrypt/CHANGELOG.md b/packages/stack-encrypt/CHANGELOG.md index 2c292ba83..bf6828727 100644 --- a/packages/stack-encrypt/CHANGELOG.md +++ b/packages/stack-encrypt/CHANGELOG.md @@ -28,6 +28,40 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `dynamic::term` take an `IndexSpec` (the last two by reference), and `Output` is no longer `Copy`. A plan's wire form is unchanged: a bare `"match"` still means the default options. +- **`dynamic::record` is a lowering into the plan builder, not a second + executor** (ADR-0007). `record::encrypt` and `record::decrypt` are no + longer `async`: each checks and converts its input with no cipher + (`Error::Source`, `Error::Term`, `Error::Record`, as before) and returns + the plan's `Pending`, whose failure is the crate's `Error`; a typed field + that opens to another kind is now `Error::Plan(PlanError::FieldType)` on + the await rather than `dynamic::Error::Record`. `dynamic::term` asks + `K: 'static`. The stored record shape (`"c"` and the term keys per field) + is unchanged, with one addition below. +- **A data plan field's `"context"` must be the field's label** — a list of + at least two plain segments (`["users", "age"]`), optionally extended by + scalar parts nested to the left (`[["users", "age"], 7]`) — and every + field of one plan must share the label's prefix and the extension, which + become the plan's one context and its `.extend(..)`. A bare text part + (`"users/age"`), an integer, bytes or a one-segment label is refused + (`Error::Plan`): a fields plan cannot seal a field outside the record + context, and the derive lost that in the same release. Two sealed or + indexed fields under one label are refused for the same reason the + builder refuses them (`PlanError::SharedIdentity`). A record a Go program + wrote with a `label=` tag is unaffected; one written with a `context=` + tag is not readable through a plan. +- **A data plan field typed `uint32` or `string` seals as a bare `u32` or + `String` leaf** — the encoding a Rust `u32` or `String` field seals, run + through the same operations — so a derived record and a data plan + interchange ciphertexts and terms under one declaration. Every other kind, + and a field with no `"type"`, seals the tagged `FfiValue` encoding, as + before. A `uint32` or `string` field written by the previous release is + therefore not readable under a plan that now declares that type; written + without a type, it reads as before. +- `dynamic::Output` gains `Passthrough` (the wire key `"passthrough"`). The + enum is exhaustive on purpose, so a match over it must name the variant. +- `target::DeclaredContext` holds several extension parts: `with(part)` + appends one, and `under` nests them to the left as `NonEmpty::with` does. + One part behaves exactly as before. - **`stack-encrypt-derive`: a field-level `#[stash(context = "..")]` is removed on every derive form** (0.2.0 accepted it). On a `plaintext = T` record, remove it: all outputs of the record share the caller's context. On @@ -98,6 +132,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `Decryption` held in a variable. - A dynamic plan's match index can carry options, as `{"match": {"tokenizer", "downcase", "k", "m"}}`. +- A dynamic plan field may be carried through: `"outputs": ["passthrough"]` + stores the value as it is under the `"passthrough"` key, unsealed and + unauthenticated, and it comes back from `decrypt`. +- `dynamic::Value`: an `FfiValue` as a plan field's plaintext, `Clone` by + deep copy into fresh `Protected` payloads, so the borrowed engine can + consume it. `dynamic::TermBytes`: a term as its frozen bytes. +- `IndexSpec` implements `Index`, `Index` and `Index`, + each with `TermBytes` as its term, so an index named as data runs through + `indexed()` like an index named as a type. `Vec` implements + `Indexes` for any `I: Index`: an index set sized at run time, whose + terms are a `Vec`; an empty one is refused when it runs + (`PlanError::EmptyIndexes`). +- `tests/fixtures/record_lowering.json`: records the typed chain and the + data-plan lowering each sealed under one declaration, with their term + bytes, under a deterministic key source; `tests/record_lowering.rs` opens + each with the other. The fixture is the proof that the two are one engine + (ADR-0007), and a Go test can read it later. - A dynamic plan field may declare its type, as `"type": ""`. The vocabulary is vitaminc's `ValueKind` (re-exported as `dynamic::ValueKind`), not a new enum, so this crate now needs vitaminc diff --git a/packages/stack-encrypt/fuzz/fuzz_targets/check_record.rs b/packages/stack-encrypt/fuzz/fuzz_targets/check_record.rs index 13c7d67ff..61edece76 100644 --- a/packages/stack-encrypt/fuzz/fuzz_targets/check_record.rs +++ b/packages/stack-encrypt/fuzz/fuzz_targets/check_record.rs @@ -60,9 +60,13 @@ impl Name { } /// A `"context"` value: the shapes `dynamic::context` accepts, plus one it -/// refuses. +/// refuses. A plan field's context must be a label (a list of at least two +/// plain segments), optionally extended, so `Label` is what parses most +/// often; the others exercise the parser's refusals. #[derive(Arbitrary, Debug)] enum Ctx { + Label(Name, Name), + Extended(Name, Name, u32), Text(Name), I64(i64), U32(u32), @@ -72,8 +76,14 @@ enum Ctx { impl Ctx { fn into_value(self) -> FfiValue { + let text = |name: Name| FfiValue::String(name.as_str().into()); match self { - Ctx::Text(name) => FfiValue::String(name.as_str().into()), + Ctx::Label(a, b) => FfiValue::Array(vec![text(a), text(b)]), + Ctx::Extended(a, b, part) => FfiValue::Array(vec![ + FfiValue::Array(vec![text(a), text(b)]), + FfiValue::UInt32(part), + ]), + Ctx::Text(name) => text(name), Ctx::I64(v) => FfiValue::Int64(v), Ctx::U32(v) => FfiValue::UInt32(v), Ctx::List(items) => FfiValue::Array(items.into_iter().map(Ctx::into_value).collect()), diff --git a/packages/stack-encrypt/src/dynamic/mod.rs b/packages/stack-encrypt/src/dynamic/mod.rs index e0891c3a9..f53246438 100644 --- a/packages/stack-encrypt/src/dynamic/mod.rs +++ b/packages/stack-encrypt/src/dynamic/mod.rs @@ -20,35 +20,44 @@ //! //! * [`context`](context()) — an [`FfiValue`] read as an encryption context. //! * [`term`](term()) — one index term for a value, dispatched on its variant. -//! * [`record`] — the runtime form of `#[derive(EncryptFrom)]`: a *plan* -//! says per field what context to bind and what outputs to produce, and -//! the whole call seals from one batched key request. +//! * [`record`] — a fields plan spelled as data, *lowered* into the plan +//! builder ([`Plan`](crate::Plan)) and run by the engine: the same code a +//! Rust chain and the derive run, so a record written from any language +//! is the same bytes (ADR-0007). The one step that stays dynamic is +//! dispatching a value whose type is known only at run time to the typed +//! term operation, which [`IndexSpec`]'s `Index` impls do. +//! * [`Value`] — an [`FfiValue`] as a plan field's plaintext, where its +//! declared type names no Rust leaf type; [`TermBytes`] — the term such a +//! field derives, as its frozen bytes. //! * [`Scope`] — which cipher an opening operation decrypts through. //! //! # What is not here //! //! Encrypting a whole value is not: [`FfiValue`] implements `Encrypt` //! already, so `keyset.encrypt(value, aad)` is the whole of it and needs -//! nothing from this module. +//! nothing from this module. Nor is a second executor: nothing here calls +//! the term functions or the seal path to produce a record. A capability a +//! data plan needs and the builder lacks is added to the builder, once. //! //! # Stability //! //! The output keys this module spells (`"c"`, `"eq"`, `"match"`, `"ore"`, -//! `"ope"`) are **wire format**, not just API: they are map keys in stored -//! ciphertext, so a row written under one spelling is read under the same -//! spelling or not at all. They are fixed here so that bindings in different -//! languages agree on them by construction rather than by each re-deriving -//! them. Their long-term home is beside vitaminc's frozen tag table, which -//! already owns this class of constant. +//! `"ope"`, `"passthrough"`) are **wire format**, not just API: they are map +//! keys in stored ciphertext, so a row written under one spelling is read +//! under the same spelling or not at all. They are fixed here so that +//! bindings in different languages agree on them by construction rather +//! than by each re-deriving them. Their long-term home is beside vitaminc's +//! frozen tag table, which already owns this class of constant. //! //! A plan field's `"type"` names (`"int64"`, `"string"`, …) are wire format //! in the same way: a binding spells them, and a stored row opens only under //! the type it was sealed as. They are not this crate's: a declared type is //! vitaminc's [`ValueKind`], re-exported here, whose names vitaminc freezes //! beside its tag table. This crate adds only what a kind means to an index -//! ([`admits`]) and to a query value ([`read`]). A field without `"type"` is -//! dispatched on each value's own tag; that is transitional, and -//! [`record::plan`] says until when. +//! ([`admits`]) and to a query value ([`read`]), and which Rust leaf type a +//! field lowers to ([`record`]). A field without `"type"` is dispatched on +//! each value's own tag; that is transitional, and [`record::plan`] says +//! until when. //! //! For the same reason the enums that spell them — [`Output`], //! [`IndexSpec`] and [`ValueKind`] — are *not* `#[non_exhaustive]`, against this workspace's @@ -60,13 +69,15 @@ mod context; mod kind; pub mod record; mod term; +mod value; use std::fmt; pub use context::{borrowed, context}; pub use kind::{admits, read}; pub use record::{FieldPlan, Output, Plan}; -pub use term::{term, Scalar}; +pub use term::{term, Scalar, TermBytes}; +pub use value::Value; /// vitaminc's language-neutral value tree — the runtime value every binding /// funnels through. Its transport codec is `vitaminc_aead_value::transport`, /// which stays the binding's: this crate takes and returns values, never @@ -129,13 +140,13 @@ fn utf8(s: &vitaminc_aead_value::Utf8String) -> Option<&str> { /// The split that matters to a caller is malformed input versus something /// else: every variant but [`Cipher`](Error::Cipher) and /// [`Internal`](Error::Internal) is a statement about the value or the -/// request, decided before any key is minted or retrieved — save one: a -/// typed field's opened value is checked against its declared type once it -/// is open (the type tag is inside the AEAD envelope), and a mismatch is -/// still [`Record`](Error::Record), a statement about the stored data. `Cipher` is the +/// request, decided before any key is minted or retrieved. `Cipher` is the /// operation failing; `Internal` is this module's own bug. A binding maps /// them to its own status codes on those lines, and must not report -/// `Internal` as the caller's fault. +/// `Internal` as the caller's fault. The record path's operations hand back +/// the engine's [`Pending`](crate::Pending), whose failure is the crate's +/// [`Error`](crate::Error); a [`Plan`](crate::Error::Plan) failure there is +/// again a statement about the caller's data. #[derive(Debug, thiserror::Error)] #[non_exhaustive] pub enum Error { @@ -161,7 +172,10 @@ pub enum Error { /// A record plan is malformed: not an object of field specs, empty, /// missing or duplicating an output, carrying a key that is not /// `"context"`, `"outputs"` or `"type"`, naming a type that is not one, - /// or asking for an index its declared type is not defined for. + /// asking for an index its declared type is not defined for, giving a + /// field a context that is not a label a fields plan can seal it under, + /// or declaring what the plan builder refuses (two fields under one + /// identity, fields under different contexts). #[error("record plan is malformed")] Plan, @@ -176,10 +190,14 @@ pub enum Error { /// A stored record does not fit its plan: not a map (or a sequence of /// them), a ciphertext-bearing field that is absent or given twice, or - /// has no `"c"` node or two of them, a repeated map key under `"c"`, or - /// a passthrough under `"c"` — which would hand back unauthenticated - /// bytes as if they had been opened — or a typed field that opens to a - /// value of another type than it declares. + /// has no `"c"` node or two of them, a repeated map key under `"c"`, a + /// passthrough under `"c"` — which would hand back unauthenticated + /// bytes as if they had been opened — or a passthrough field that is + /// absent, carries no `"passthrough"` node, or carries a value of + /// another type than it declares. A sealed field that opens to a value + /// of another type than it declares fails the pending instead + /// ([`PlanError::FieldType`](crate::PlanError::FieldType)): the type tag + /// is inside the AEAD envelope. #[error("stored record does not fit the plan")] Record, diff --git a/packages/stack-encrypt/src/dynamic/record.rs b/packages/stack-encrypt/src/dynamic/record.rs index 6040a57ae..9e523e01b 100644 --- a/packages/stack-encrypt/src/dynamic/record.rs +++ b/packages/stack-encrypt/src/dynamic/record.rs @@ -1,62 +1,79 @@ -//! Records: the runtime form of `#[derive(EncryptFrom)]`. +//! Records from data: a plan spelled as a value, lowered into the plan +//! builder. //! -//! A *plan* says, per field, which encryption context to bind and which -//! outputs to produce; the source supplies the field values. That is the -//! same job the derive does from a struct definition, done from data — which -//! is all a binding has. +//! A binding has no types to name, so it declares a record as data: per +//! field, a context, what to produce and, optionally, what type the values +//! are. This module reads that declaration ([`plan`]) and *lowers* it into +//! the same [`Plan`](crate::Plan) a Rust chain writes and the derive emits: +//! `Plan::context(c).fields()`, then `encrypt`, `encrypt_index`, `index` or +//! `passthrough` per field. Encrypting runs that plan's description through +//! [`KeysetCipher::run`](crate::KeysetCipher::run); decrypting runs its +//! opener. There is no second executor here: every field's context, every +//! key request and the one batch they settle in are the engine's, so a +//! record written from Go and one written from Rust under the same +//! declaration are the same bytes because they ran the same code (ADR-0007). //! -//! However many rows and fields are in one call, all ciphertext leaves seal -//! from **one** batched `generate_keys`: the pendings are merged before -//! settling, exactly like the derive's `zip`/`all` composition. Index terms -//! are *not* in that batch — [`encrypt`] settles each term as it builds the -//! row, which under the local HMAC backend is no ZeroKMS traffic at all, and -//! under a backend that derives terms at ZeroKMS would be one round trip per -//! term until the term pendings are merged into the row's batch. That is a -//! change for this module when such a backend lands, not something the -//! record path promises today. +//! What stays dynamic is one step: a field whose type is known only when its +//! value arrives dispatches to the typed term operation then, through +//! [`IndexSpec`]'s [`Index`] impls (`scalar_term` in the `term` module is the +//! one table). //! -//! # One context per field, both halves +//! # What a field's `"context"` must be //! -//! A field's context is proven [`NonEmpty`] once, when the plan is built, -//! and one borrowed view of it — a single local in the row builder — drives -//! the field's ciphertext and every one of its terms. That is ADR-0004's -//! property. The typed path holds it with a type parameter threaded through -//! the declaration tree; this path has no tree to thread, sealing through -//! the cipher-directed `encrypt_with_aad` instead, so it holds it by one -//! variable: [`encrypt`] never has two contexts for a field in hand, so it -//! cannot seal the value under one and index it under another. That is -//! enforcement by shape rather than by type, and the tests here pin it — a -//! record's `"c"` opens under its plan context and its terms equal the -//! standalone derivation under that same context. +//! A plan has **one** context, and every field is sealed under +//! `/`; a declaration gives each field its whole label, +//! as a database column is named: `["users", "age"]`. The lowering reads the +//! label's last segment as the field's identity and the rest as the plan's +//! context, so every field of one plan must share that prefix, and a label +//! has at least two segments. A field's label may be extended by the +//! caller's parts, nested to the left as a binding extends one part at a +//! time (`[["users", "age"], 7]`, then `[[["users", "age"], 7], "eu"]`), and +//! every field must carry the same extension: it becomes the call's +//! `.extend(..)`. A context that is not a label (one text part `"users/age"`, +//! an integer, bytes, a one-element list) is refused as a plan a fields plan +//! cannot express. //! -//! A plan context is the *whole* context of its field. There is no caller -//! context to extend it with, so the plan spells the extension itself: a -//! bare string matches a Rust record sealed with `encrypt_into` (no caller -//! context); a list matches one sealed with `encrypt_into_with_context` — -//! see [`super::context`](super::context()) for which list spells which Rust -//! context. Rows are readable across the two however they were sealed, -//! provided the plan names the context the row was sealed under. +//! # What a field's `"type"` decides //! -//! # Terms ride as passthrough +//! The declared type is the data form of the Rust chain's `::`. A field +//! typed `uint32` or `string` lowers to a `u32` or `String` field — read out +//! of the value as that type, sealed and indexed through exactly the +//! operations `encrypt_index::` runs — so a `uint32` field of a Go +//! record and a `u32` field of a Rust record interchange, ciphertext and +//! terms alike. Every other kind, and a field with no `"type"`, is a +//! [`Value`]: it seals in vitaminc's self-describing tagged leaf encoding +//! (`[tag] ++ payload`), which only a dynamic reader opens, and its terms +//! dispatch on each value's own variant. The kinds with no bare Rust leaf +//! type (`uint64`, `int32`, `bool`, the floats, …) have no other encoding to +//! take; a field with no type has no type to name, which is transitional +//! (#1082). //! -//! A term is a comparand, not a ciphertext to open, and passthrough is its -//! honest encoding: the result tree carries each term as a -//! [`CipherText::Passthrough`] byte node beside the field's `"c"` subtree. -//! Under `"c"` itself a passthrough is refused in both directions, and that -//! is load-bearing: `decrypt_as` collects **zero** retrieve-requests for a -//! passthrough and returns its payload with no AEAD opened, so without the -//! decrypt-side refusal an attacker with write access to the stored tree -//! could replace a field's `"c"` subtree with a passthrough carrying forged -//! plaintext and have it reported as a successful decrypt. - -use stack_kms::DataKeySource; +//! # The stored record is wire format +//! +//! A record is stored as `field → { output-key → node }`: `"c"` is the +//! field's ciphertext, each term rides under its index key (`"eq"`, +//! `"match"`, `"ore"`, `"ope"`) as a passthrough byte node, and a passthrough +//! field rides under `"passthrough"`. A row written under one spelling is +//! read under the same spelling or not at all, so the keys are fixed here +//! and every binding agrees on them by construction. +//! +//! Under `"c"` a passthrough is refused in both directions, and that is +//! load-bearing: opening a passthrough retrieves no key and opens no AEAD, +//! it hands the payload back — so without the decrypt-side refusal an +//! attacker with write access to the stored tree could replace a field's +//! `"c"` subtree with a passthrough carrying forged plaintext and have it +//! reported as a successful decrypt. The encrypt-side refusal is what makes +//! that a round-trip invariant rather than data loss. + use vitaminc_aead_value::{FfiValue, ValueKind}; -use vitaminc_protected::Protected; +use vitaminc_protected::Controlled; -use super::{admits, borrowed, term, utf8, Error, Scalar, Scope}; -use crate::target::{IndexSpec, Pending}; +use super::{admits, utf8, Error, Scalar, Scope, TermBytes, Value}; +use crate::plan::{FieldValues, FieldsBuilder, Opens, Runs}; +use crate::target::{CallerContext, DeclaredContext, Decryption, Encrypted, Index, IndexSpec}; use crate::{ - BoxedPassthrough, CipherText, ContextPiece, Encrypt, KeysetCipher, NonEmpty, StackCipherText, + BoxedPassthrough, CipherText, ContextPiece, KeysetCipher, Label, NonEmpty, Pending, + StackCipherText, }; /// What a plan field asks for. @@ -71,6 +88,10 @@ pub enum Output { /// An index term, keyed `"eq"`, `"match"`, `"ore"` or `"ope"`: the /// index, with its options, that derives it. Term(IndexSpec), + /// `"passthrough"` — the field carried as it is, **unsealed and + /// unauthenticated**, so the record is whole. A field's only output + /// when it has it. + Passthrough, } impl Output { @@ -81,12 +102,13 @@ impl Output { pub fn parse(s: &str) -> Option { match s { "c" => Some(Output::Ciphertext), + "passthrough" => Some(Output::Passthrough), _ => IndexSpec::parse(s).map(Output::Term), } } - /// Read one entry of a plan's `"outputs"` list: `"c"`, or an index in - /// its wire form ([`IndexSpec::from_value`]; see [`plan`]). + /// Read one entry of a plan's `"outputs"` list: `"c"`, `"passthrough"`, + /// or an index in its wire form ([`IndexSpec::from_value`]; see [`plan`]). /// /// # Errors /// @@ -94,6 +116,7 @@ impl Output { pub fn from_value(value: &FfiValue) -> Result { match value { FfiValue::String(s) if utf8(s) == Some("c") => Ok(Output::Ciphertext), + FfiValue::String(s) if utf8(s) == Some("passthrough") => Ok(Output::Passthrough), _ => IndexSpec::from_value(value).map(Output::Term), } } @@ -103,25 +126,56 @@ impl Output { match self { Output::Ciphertext => "c", Output::Term(kind) => kind.key(), + Output::Passthrough => "passthrough", + } + } +} + +/// The verb a field's outputs lower to: one of the plan builder's four. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +enum Verb { + Encrypt, + EncryptIndex, + Index, + Passthrough, +} + +/// The Rust plaintext type a field lowers to, from its declared kind. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +enum Leaf { + /// `"uint32"`: a `u32`, as a Rust chain's `encrypt_index::` field. + U32, + /// `"string"`: a `String`. + Text, + /// Any other kind, or none: a [`Value`], the tagged leaf encoding. + Value, +} + +impl Leaf { + fn of(kind: Option) -> Self { + match kind { + Some(ValueKind::UInt32) => Leaf::U32, + Some(ValueKind::String) => Leaf::Text, + _ => Leaf::Value, } } } -/// One field of a record plan: what to call it, what context to bind it +/// One field of a record plan: what to call it, what label to seal it /// under, what to produce for it, and, optionally, what type its values are. /// /// A field with a declared type (a [`ValueKind`]) admits only the indexes -/// that kind is defined for ([`admits`], checked when the -/// plan is built), seals only values of that kind and opens only to one -/// (checked per value), so the engine verifies what a binding hands it -/// rather than trusting the binding's tagging. A field with no declared type -/// is dispatched on each value's own type, as every field was before types -/// existed; that keeps the plans existing bindings send valid, and is -/// transitional (see [`plan`]). +/// that kind is defined for ([`admits`], checked when the plan is built), +/// seals only values of that kind and opens only to one (checked per value), +/// so the engine verifies what a binding hands it rather than trusting the +/// binding's tagging. A field with no declared type is dispatched on each +/// value's own type; see the [module docs](self#what-a-fields-type-decides). #[derive(Clone, Debug)] pub struct FieldPlan { name: String, context: NonEmpty>, + label: Label, + extension: Vec>, outputs: Vec, field_type: Option, } @@ -129,19 +183,20 @@ pub struct FieldPlan { impl FieldPlan { /// A field plan. /// - /// The context is a proven [`NonEmpty`] because that proof has to happen - /// somewhere and here is the last place it can: the cipher-directed path - /// [`encrypt`] seals through accepts any AAD, so nothing downstream - /// would stop an empty context from being sealed under — and opening - /// goes through `decrypt_as`, which would then never open it. Build one - /// from a value with [`super::context`](super::context()). + /// `context` is the field's whole context as a binding spells it: its + /// label, a list of at least two plain segments, extended by zero or more + /// scalar parts nested to the left (see the + /// [module docs](self#what-a-fields-context-must-be)). Build one from a + /// value with [`super::context`](super::context()). /// /// # Errors /// - /// [`Error::Plan`] if `outputs` is empty or names an output twice. Two + /// [`Error::Plan`] if `outputs` is empty, names an output twice (two /// outputs with the same [key](Output::key) are the same output — two /// match indexes under different options would both ride under - /// `"match"` — so they are refused too. + /// `"match"`), or names [`Output::Passthrough`] beside another output; + /// or if `context` is not a label of at least two segments, optionally + /// extended. pub fn new( name: impl Into, context: NonEmpty>, @@ -158,9 +213,18 @@ impl FieldPlan { return Err(Error::Plan); } } + if outputs.contains(&Output::Passthrough) && outputs.len() > 1 { + return Err(Error::Plan); + } + let (label, extension) = split_context(context.get())?; + if label.segments().len() < 2 { + return Err(Error::Plan); + } Ok(Self { name: name.into(), context, + label, + extension, outputs, field_type: None, }) @@ -171,8 +235,8 @@ impl FieldPlan { /// # Errors /// /// [`Error::Plan`] if the field asks for an index the kind is not - /// defined for ([`admits`]): match on an integer, - /// equality on a float, any index on a composite. + /// defined for ([`admits`]): match on an integer, equality on a float, + /// any index on a composite. pub fn with_type(mut self, field_type: ValueKind) -> Result { for output in &self.outputs { if let Output::Term(index) = output { @@ -190,11 +254,26 @@ impl FieldPlan { &self.name } - /// The context this field binds under, on both halves. + /// The field's whole context as it was declared: its label, extended + /// by the call's parts if any. What a probe for the field takes. pub fn context(&self) -> &NonEmpty> { &self.context } + /// The label the field is sealed and indexed under before any + /// extension: the plan's context, then the field's identity. + pub fn label(&self) -> &Label { + &self.label + } + + /// The label segment the field's data is keyed under: the last segment + /// of its label. + pub fn identity(&self) -> &str { + // A label has at least two segments by construction (`new`), so + // this never falls back. + self.label.segments().last().unwrap_or("") + } + /// What the field produces. pub fn outputs(&self) -> &[Output] { &self.outputs @@ -212,28 +291,115 @@ impl FieldPlan { self.outputs.contains(&Output::Ciphertext) } - /// A borrowed view of the context, so one proof serves every output of - /// every row without copying the payloads. - fn view(&self) -> Result>, Error> { - // The proof was made when the plan was built, so re-taking it over - // the same tree cannot fail. - NonEmpty::new(borrowed(self.context.get())).map_err(|_| Error::Internal) + /// Whether the field comes back from [`decrypt`]: sealed and passthrough + /// fields do, index-only fields do not. + fn opens(&self) -> bool { + self.verb() != Verb::Index + } + + fn verb(&self) -> Verb { + if self.outputs.contains(&Output::Passthrough) { + Verb::Passthrough + } else if self.has_ciphertext() { + if self.indexes().is_empty() { + Verb::Encrypt + } else { + Verb::EncryptIndex + } + } else { + Verb::Index + } + } + + fn leaf(&self) -> Leaf { + Leaf::of(self.field_type) + } + + /// The indexes the field declares, in output order. + fn indexes(&self) -> Vec { + self.outputs + .iter() + .filter_map(|output| match output { + Output::Term(index) => Some(index.clone()), + Output::Ciphertext | Output::Passthrough => None, + }) + .collect() + } + + /// The plan context the field's label sits under: every segment but + /// the last. + fn prefix(&self) -> Result { + let segments: Vec<&str> = self.label.segments().collect(); + let Some((_, prefix)) = segments.split_last() else { + return Err(Error::Internal); + }; + Label::new(prefix).map_err(|_| Error::Plan) + } + + /// What the output adapters need of the field: no context, which is the + /// engine's by then. + fn shape(&self) -> FieldShape { + FieldShape { + name: self.name.clone(), + verb: self.verb(), + leaf: self.leaf(), + keys: self.indexes().iter().map(IndexSpec::key).collect(), + kind: self.field_type, + } + } +} + +/// The text of a text part. +fn text_of<'a>(piece: &'a ContextPiece<'_>) -> Option<&'a str> { + match piece { + ContextPiece::Text(text) => Some(text.as_ref()), + _ => None, + } +} + +/// A field's declared context, taken apart into its label and the +/// extension parts around it. +/// +/// A list of plain text segments is the label. A two-element list whose +/// second element is a scalar is a context extended by that part, nested to +/// the left, so the first element is taken apart in turn. Anything else — a +/// bare part, a one-element list, a list mixing segments and other parts, a +/// part that is itself a list — is not a context a fields plan can give a +/// field. +fn split_context(piece: &ContextPiece<'_>) -> Result<(Label, Vec>), Error> { + let ContextPiece::List(parts) = piece else { + return Err(Error::Plan); + }; + if let Some(segments) = parts.iter().map(text_of).collect::>>() { + if segments.len() >= 2 { + let label = Label::new(segments).map_err(|_| Error::Plan)?; + return Ok((label, Vec::new())); + } + } + match parts.as_slice() { + [inner, part] if !matches!(part, ContextPiece::List(_)) => { + let (label, mut extension) = split_context(inner)?; + extension.push(part.clone().into_owned()); + Ok((label, extension)) + } + _ => Err(Error::Plan), } } /// A record plan: the fields a record has, each with what to call it, what -/// context to bind it under, and what to produce for it. +/// label to seal it under, and what to produce for it. /// -/// Opaque, because the operations over a plan rely on two properties of the +/// Opaque, because the operations over a plan rely on properties of the /// whole that no single [`FieldPlan`] can carry: there is at least one -/// field, and no two fields share a name. With a repeated name the source -/// check would accept a row that names the field once, and [`encrypt`] -/// would write a map with the same key twice — a stored record no reader -/// can take apart. Both the parser ([`plan`]) and the manual constructor -/// ([`Plan::new`]) go through the one check, so a plan in hand is a plan -/// that holds them, whichever way it was built. +/// field, no two fields share a name, every field's label sits under the +/// one plan context and carries the one extension, and the whole lowers to +/// a [`Plan`](crate::Plan) that builds. Both the parser ([`plan`]) and the +/// manual constructor ([`Plan::new`]) go through the one check, so a plan in +/// hand is a plan that holds them, whichever way it was built. #[derive(Clone, Debug)] pub struct Plan { + context: Label, + extension: Vec>, fields: Vec, } @@ -243,17 +409,36 @@ impl Plan { /// /// # Errors /// - /// [`Error::Plan`] if `fields` is empty or names a field twice. + /// [`Error::Plan`] if `fields` is empty, names a field twice, has + /// fields whose labels sit under different contexts or carry different + /// extensions, or does not build as a fields plan: two sealed or indexed + /// fields keyed under one identity, for instance, whose terms would be + /// interchangeable. pub fn new(fields: Vec) -> Result { - if fields.is_empty() { + let Some(first) = fields.first() else { return Err(Error::Plan); - } + }; + let context = first.prefix()?; + let extension = first.extension.clone(); for (at, field) in fields.iter().enumerate() { if fields[..at].iter().any(|prior| prior.name == field.name) { return Err(Error::Plan); } + if field.prefix()? != context || field.extension != extension { + return Err(Error::Plan); + } } - Ok(Self { fields }) + let plan = Self { + context, + extension, + fields, + }; + // The whole-plan rules the builder holds (names once, labels plain, + // no shared identity) are checked by building, so a plan in hand + // lowers. `()` stands in for the key source: the check does not + // depend on it. + let _ = plan.lower::<()>().map_err(|_| Error::Plan)?; + Ok(plan) } /// The plan's fields, in result order. Never empty, and no two share a @@ -261,6 +446,71 @@ impl Plan { pub fn fields(&self) -> &[FieldPlan] { &self.fields } + + /// The plan's one context: the label every field's label extends. + pub fn label(&self) -> &Label { + &self.context + } + + /// The parts every field's label is extended by, in order; empty when + /// the declaration carries none. + pub fn extension(&self) -> &[ContextPiece<'static>] { + &self.extension + } + + /// The fields plan this declaration lowers to, for a cipher over `K`. + /// + /// Built afresh per call: a built plan is bound to its key source type, + /// and a declaration is not. Every field is declared by name, read out of + /// the [`FieldValues`] the source is converted into, at the Rust type its + /// kind lowers to. + fn lower(&self) -> Result, crate::Error> { + let mut builder = crate::Plan::context(self.context.clone()).fields::(); + for field in &self.fields { + builder = match field.leaf() { + Leaf::U32 => declare::(builder, field), + Leaf::Text => declare::(builder, field), + Leaf::Value => declare::(builder, field), + }; + if field.identity() != field.name { + builder = builder.identity(field.identity()); + } + } + builder.build() + } + + /// The extension every field's label is run under, as the chain's + /// `.extend(..)` would carry it. + fn declared_context(&self) -> DeclaredContext { + self.extension + .iter() + .cloned() + .fold(DeclaredContext::default(), |context, part| { + context.with(CallerContext::from_piece(part)) + }) + } + + fn shape(&self) -> Vec { + self.fields.iter().map(FieldPlan::shape).collect() + } +} + +/// One field's verb, at the type its kind lowers to. +fn declare( + builder: FieldsBuilder, + field: &FieldPlan, +) -> FieldsBuilder +where + F: crate::Encrypt + crate::Decrypt<'static> + Clone + Send + 'static, + IndexSpec: Index, +{ + let name = field.name.as_str(); + match field.verb() { + Verb::Encrypt => builder.encrypt::(name), + Verb::EncryptIndex => builder.encrypt_index::(name, field.indexes()), + Verb::Index => builder.index::(name, field.indexes()), + Verb::Passthrough => builder.passthrough::(name), + } } /// Read a record plan from a decoded value. @@ -268,7 +518,7 @@ impl Plan { /// The plan is an [`FfiValue::Object`]: /// /// ```text -/// { : { "context": , "outputs": [ "c" | , ... ], "type": }, ... } +/// { : { "context": , "outputs": [ "c" | "passthrough" | , ... ], "type": }, ... } /// ``` /// /// `` is an index in its wire form, which is its key — `"eq"`, @@ -289,7 +539,7 @@ impl Plan { /// other three indexes have no options and no object form. A field names /// each output key at most once, so it carries at most one match index. /// [`IndexSpec::to_value`] writes this form and [`IndexSpec::from_value`] -/// reads it. +/// reads it. `"passthrough"` is a field's only output when it has it. /// /// [`MatchOptions::default`]: crate::sem::MatchOptions::default /// @@ -297,7 +547,8 @@ impl Plan { /// …; see [`ValueKind::name`]): vitaminc's vocabulary, not one of this /// crate's. Declared, it is checked against the field's outputs here /// ([`admits`]) and against every value sealed into or opened -/// from the field. +/// from the field, and it decides the field's leaf encoding (see the +/// [module docs](self#what-a-fields-type-decides)). /// /// **An indexed field without `"type"` is dispatched on each value's own /// tag**, so for that field the engine trusts the binding to tag every value @@ -308,9 +559,9 @@ impl Plan { /// types; then `"type"` becomes required on every field with a term output /// (#1082). /// -/// `` is defined once, in [`super::context`](super::context()): a -/// string, bytes, an integer, or a list of those, with what each spells in -/// Rust and the emptiness rule. +/// `` is read by [`super::context`](super::context()) and must be +/// the field's label, optionally extended; the +/// [module docs](self#what-a-fields-context-must-be) give the shape. /// /// # Examples /// @@ -319,7 +570,7 @@ impl Plan { /// use stack_encrypt::target::IndexSpec; /// /// // As a binding would decode it from its caller: seal `age` under the -/// // pair ("users", "age") and index it for equality. +/// // label users/age and index it for equality. /// let plan = record::plan(FfiValue::Object(vec![( /// "age".to_string(), /// FfiValue::Object(vec![ @@ -342,6 +593,7 @@ impl Plan { /// /// assert_eq!(plan.fields().len(), 1); /// assert_eq!(plan.fields()[0].name(), "age"); +/// assert_eq!(plan.label().to_string(), "users"); /// assert_eq!( /// plan.fields()[0].outputs(), /// [Output::Ciphertext, Output::Term(IndexSpec::Equality)] @@ -354,12 +606,15 @@ impl Plan { /// [`Error::Plan`] for a plan that is not an object of field specs, an /// empty plan, a field named twice, a spec with a key other than /// `"context"`, `"outputs"` and `"type"` or with one given twice, missing -/// `"context"` or `"outputs"`, an output list that is not a list of -/// outputs (`"c"` or an index in its wire form, above), is empty, or names -/// an output key twice, a `"type"` that is not -/// a string naming a [`ValueKind`], or a type that does not admit one of -/// the field's index outputs. [`Error::Context`] for a `"context"` that is -/// present but is not a context, or renders empty. +/// `"context"` or `"outputs"`, an output list that is not a list of outputs +/// (above), is empty, names an output key twice or names `"passthrough"` +/// beside another output, a `"type"` that is not a string naming a +/// [`ValueKind`], a type that does not admit one of the field's index +/// outputs, a context that is not a label of at least two segments +/// (optionally extended), fields under different contexts or extensions, +/// or a plan the builder refuses ([`Plan::new`]). [`Error::Context`] for a +/// `"context"` that is present but is not a context at all, or renders +/// empty. /// /// The transport codec refuses duplicate object keys before a binding's /// value reaches here, but an [`FfiValue`] can be built with them directly @@ -411,22 +666,31 @@ pub fn plan(value: FfiValue) -> Result { None => field, }); } - // The whole-plan rules — non-empty, no name twice — are `Plan::new`'s, - // so a parsed plan and a hand-built one are refused alike. + // The whole-plan rules are `Plan::new`'s, so a parsed plan and a + // hand-built one are refused alike. Plan::new(fields) } /// Encrypt a record — or a batch of records — per a plan. /// -/// `source` is an [`FfiValue::Object`] of `{ field: scalar }` (one record), +/// `source` is an [`FfiValue::Object`] of `{ field: value }` (one record), /// or an [`FfiValue::Array`] of such objects (a batch). Every plan field /// must be present in each record, and every record field must be named by /// the plan — silently dropping a field on either side would lose data or /// index nothing. /// -/// The result is per record a map of `field → { output-key → node }`, where -/// `"c"` is the field's sealed ciphertext subtree and each term rides as a -/// passthrough byte node. A batch is a sequence of such maps. +/// The source is checked and converted here, with no cipher: that is +/// [`check_source`], and it is where [`Error::Source`] and [`Error::Term`] +/// come from. What comes back is the plan's [`Pending`], with every term +/// derived and every key request queued and nothing sent: one batched +/// `generate_keys` for every ciphertext leaf of every row when it is +/// awaited, however many rows and fields there are. Its failure is the +/// engine's [`Error`](crate::Error). +/// +/// The result is per record a map of `field → { output-key → node }` (see +/// the [module docs](self#the-stored-record-is-wire-format)): `"c"` first, +/// then each term in the order the plan named its indexes. A batch is a +/// sequence of such maps. /// /// # Examples /// @@ -442,7 +706,7 @@ pub fn plan(value: FfiValue) -> Result { /// .await?; /// let keyset = cipher.default_keyset(); /// -/// // Seal `age` under the pair ("users", "age") with an equality term beside it. +/// // Seal `age` under users/age with an equality term beside it. /// let plan = record::plan(FfiValue::Object(vec![( /// "age".to_string(), /// FfiValue::Object(vec![ @@ -464,10 +728,10 @@ pub fn plan(value: FfiValue) -> Result { /// )]))?; /// /// let row = FfiValue::Object(vec![("age".to_string(), FfiValue::UInt32(34))]); -/// let sealed = record::encrypt(&keyset, row, &plan).await?; +/// let sealed = record::encrypt(&keyset, row, &plan)?.await?; /// /// // Only the ciphertext comes back; the term is one-way. -/// let opened = record::decrypt(Scope::Client(&cipher), sealed, &plan).await?; +/// let opened = record::decrypt(Scope::Client(&cipher), sealed, &plan)?.await?; /// let FfiValue::Object(fields) = opened else { /// unreachable!("one record opens to one object"); /// }; @@ -476,212 +740,160 @@ pub fn plan(value: FfiValue) -> Result { /// # }).unwrap(); /// ``` /// -/// # Cross-language note -/// -/// A `"c"` leaf seals the aead-value *tagged* plaintext encoding (`[type -/// tag] ++ payload`), because that tag table is the contract the bindings -/// share. A Rust `#[derive(EncryptFrom)]` over a plain primitive — a bare -/// `u32` — seals four untagged bytes instead, so a plain-primitive Rust -/// derive and a plan do **not** interchange ciphertexts for the same field -/// until the Rust side uses aead-value's tagged types too. This is by -/// design, not a defect in either side. -/// /// # Errors /// /// [`Error::Source`] if the source does not fit the plan; [`Error::Term`] -/// if a value has no term the plan asks for; [`Error::Cipher`] if sealing -/// or deriving fails. -pub async fn encrypt( - cipher: &KeysetCipher<'_, K>, +/// if a value has no term the plan asks for. Both are decided here, before +/// the pending exists. A failure of the pending itself is the engine's. +pub fn encrypt<'a, K: 'static>( + cipher: &'a KeysetCipher<'_, K>, source: FfiValue, plan: &Plan, -) -> Result -where - K: DataKeySource + Sync, -{ - let Rows { rows, batched } = source_rows(source, plan)?; - - // Build every row: terms derive now (local), ciphertexts queue their - // data-key requests into one flat pending list. - let mut pendings: Vec> = Vec::new(); - let mut skeletons: Vec> = Vec::with_capacity(rows.len()); - for row in rows { - skeletons.push(build_row(cipher, row, plan, &mut pendings).await?); - } - - // The one batched key request for the whole invocation. - let mut sealed = Settled::of(Pending::all(cipher, pendings).await?); - - // Fill the ciphertext slots back in, in build order. - let row_nodes = skeletons - .into_iter() - .map(|skeleton| { - let fields = skeleton - .into_iter() - .map(|field| { - let nodes = field - .outputs - .into_iter() - .map(|(key, slot)| { - let node = match slot { - Slot::Term(term) => CipherText::Passthrough(Box::new( - FfiValue::Bytes(Protected::new(term)), - ) - as BoxedPassthrough), - Slot::Ciphertext => sealed.next()?, - }; - Ok((key.to_string(), node)) - }) - .collect::, Error>>()?; - Ok((field.name, CipherText::Map(nodes))) - }) - .collect::, Error>>()?; - Ok(CipherText::Map(fields)) - }) - .collect::, Error>>()?; - sealed.finish()?; - - Rows { - rows: row_nodes, - batched, - } - .reshape(CipherText::Sequence) +) -> Result, Error> { + let rows = source_rows(source, plan)?; + let lowered = plan.lower::().map_err(|_| Error::Internal)?; + let extend = plan.declared_context(); + let shape = plan.shape(); + Ok(match rows { + Rows::One(values) => { + Runs::::pending(&lowered, cipher, &values, None, extend) + .try_map(move |values| shape_record(values, &shape)) + } + Rows::Batch(rows) => Runs::<[FieldValues], K>::pending( + &lowered, cipher, &rows, None, extend, + ) + .try_map(move |rows| { + rows.into_iter() + .map(|values| shape_record(values, &shape)) + .collect::, _>>() + .map(CipherText::Sequence) + }), + }) } /// Decrypt a record — or a batch — produced by [`encrypt`] under the same /// plan. /// -/// Only the `"c"` outputs participate: terms are one-way. The result is an -/// [`FfiValue::Object`] per record holding the plan's ciphertext-bearing -/// fields, in plan order — or an [`FfiValue::Array`] of them for a batch. -/// One batched `retrieve_keys` per invocation and, when opening through -/// [`Scope::Client`], one per keyset the leaves were sealed under. +/// Only the `"c"` and `"passthrough"` outputs participate: terms are +/// one-way. The stored tree is checked and read here, with no cipher: that +/// is [`check_record`], and it is where [`Error::Record`] comes from. What +/// comes back is the plan's opener as a [`Pending`]: one batched +/// `retrieve_keys` for every ciphertext leaf when it is awaited and, when +/// opening through [`Scope::Client`], one per keyset the leaves were sealed +/// under. Its value is an [`FfiValue::Object`] per record holding the plan's +/// fields that come back, in plan order — or an [`FfiValue::Array`] of them +/// for a batch. /// /// # Errors /// -/// [`Error::Record`] if the stored tree does not fit the plan; -/// [`Error::Cipher`] if opening fails — including the expected outcome for -/// a wrong context, a wrong key, a tampered ciphertext, or a leaf from a -/// keyset other than a [`Scope::Keyset`]'s. -pub async fn decrypt( - scope: Scope<'_, K>, +/// [`Error::Record`] if the stored tree does not fit the plan, decided +/// here. A failure of the pending is the engine's: a wrong context, a wrong +/// key, a tampered ciphertext, a leaf from a keyset other than a +/// [`Scope::Keyset`]'s ([`Error::ForeignKeyset`](crate::Error::ForeignKeyset), +/// before any key is retrieved), or a typed field that opens to a value of +/// another kind than it declares ([`PlanError::FieldType`](crate::PlanError::FieldType) +/// — the type tag is inside the AEAD envelope, so only opening can see it). +pub fn decrypt<'a, K: 'static>( + scope: Scope<'a, K>, record: StackCipherText, plan: &Plan, -) -> Result -where - K: DataKeySource + Sync + 'static, -{ - let Rows { rows, batched } = record_leaves(record, plan)?; - let opened = plan - .fields - .iter() - .filter(|field| field.has_ciphertext()) - .collect::>(); - let contexts = opened - .iter() - .map(|field| field.view()) - .collect::, Error>>()?; - - // Per row, per ciphertext-bearing plan field (in plan order, as - // `record_leaves` lifted them): queue the "c" subtree's decrypt. - let mut pendings: Vec> = Vec::new(); - let mut names: Vec> = Vec::with_capacity(rows.len()); - for row in rows { - if row.len() != contexts.len() { - return Err(Error::Internal); - } - let mut row_names = Vec::with_capacity(row.len()); - for ((name, ct), context) in row.into_iter().zip(&contexts) { - let context = context.clone(); - // The scope is the caller's, the declaration is the target's: - // `decrypt_as` takes one context and drives both halves with it. - pendings.push(match &scope { - Scope::Client(cipher) => cipher.decrypt_as(ct, context.into()), - Scope::Keyset(keyset) => keyset.decrypt_as(ct, context.into()), - }); - row_names.push(name); - } - names.push(row_names); - } - - // The one batched key request for the whole invocation. - let mut values = Settled::of(match &scope { - Scope::Client(cipher) => Pending::all(*cipher, pendings).await, - Scope::Keyset(keyset) => Pending::all(keyset, pendings).await, - }?); - - let row_values = names - .into_iter() - .map(|row_names| { - let entries = row_names - .into_iter() - .zip(&opened) - .map(|(name, field)| { - let value = values.next()?; - // The tag is authenticated, so a mismatch is not - // tampering: the row was sealed as another type than - // the plan now declares. - match field.field_type { - Some(declared) if !declared.holds(&value) => Err(Error::Record), - _ => Ok((name, value)), - } - }) - .collect::, Error>>()?; - Ok(FfiValue::Object(entries)) - }) - .collect::, Error>>()?; - values.finish()?; +) -> Result, Error> { + let rows = record_rows(record, plan)?; + let lowered = plan.lower::().map_err(|_| Error::Internal)?; + let extend = plan.declared_context(); + let shape = plan.shape(); + Ok(match rows { + Rows::One(values) => run( + scope, + Opens::::decryption(&lowered, values, None, extend), + ) + .try_map(move |values| open_record(values, &shape)), + Rows::Batch(rows) => run( + scope, + Opens::, K>::decryption(&lowered, rows, None, extend), + ) + .try_map(move |rows| { + rows.into_iter() + .map(|values| open_record(values, &shape)) + .collect::, _>>() + .map(FfiValue::Array) + }), + }) +} - Rows { - rows: row_values, - batched, +/// Run an opener through the scope the caller chose: the client opens +/// leaves from any of its keysets, a keyset cipher refuses a foreign one +/// before any key is retrieved. +fn run<'a, K: 'static, T: 'static>( + scope: Scope<'a, K>, + decryption: Decryption, +) -> Pending<'a, T, K> { + match scope { + Scope::Client(cipher) => cipher.run_decryption(decryption), + Scope::Keyset(keyset) => keyset + .cipher() + .run_decryption(decryption) + .scoped_to(keyset.keyset_id()), } - .reshape(FfiValue::Array) } /// Check a source against a plan without encrypting it — everything -/// [`encrypt`] checks before it consults the cipher. +/// [`encrypt`] checks before it builds its pending. /// /// A binding runs this at its boundary so a malformed call fails the same /// way whether or not a cipher is available, and never costs a keyset load. -/// It is the same parser [`encrypt`] runs, so the two cannot disagree on -/// what is malformed. +/// It is the same conversion [`encrypt`] runs, followed by the lowered +/// plan's own check of the value, so the two cannot disagree on what is +/// malformed. /// /// # Errors /// /// As [`encrypt`], minus the cipher. pub fn check_source(source: FfiValue, plan: &Plan) -> Result<(), Error> { - source_rows(source, plan).map(drop) + let rows = source_rows(source, plan)?; + let lowered = plan.lower::<()>().map_err(|_| Error::Internal)?; + let check = |values: &FieldValues| { + Runs::::check(&lowered, values, None).map_err(|_| Error::Source) + }; + match &rows { + Rows::One(values) => check(values), + Rows::Batch(rows) => rows.iter().try_for_each(check), + } } /// Check a stored record against a plan without opening it — everything -/// [`decrypt`] checks before it consults the cipher. See [`check_source`]. +/// [`decrypt`] checks before it builds its pending. See [`check_source`]. /// /// A typed field's declared type is not among them: the type is the sealed -/// leaf's tag, inside the AEAD envelope, so only [`decrypt`] can check it. +/// leaf's tag, inside the AEAD envelope, so only awaiting [`decrypt`] can +/// check it. /// /// # Errors /// /// As [`decrypt`], minus the cipher. pub fn check_record(record: StackCipherText, plan: &Plan) -> Result<(), Error> { - record_leaves(record, plan).map(drop) + let rows = record_rows(record, plan)?; + let lowered = plan.lower::<()>().map_err(|_| Error::Internal)?; + let check = |values: &FieldValues| { + Opens::::check(&lowered, values, None).map_err(|_| Error::Record) + }; + match &rows { + Rows::One(values) => check(values), + Rows::Batch(rows) => rows.iter().try_for_each(check), + } } // ============================================================================= -// The two trees a record path walks +// The two trees a record path reads // ============================================================================= -/// The two trees a record path walks — a source ([`FfiValue`]) and a stored -/// record ([`StackCipherText`]) — seen the one way the path needs to see -/// them: as one row (a map of named nodes) or a sequence of rows, and as a -/// tree that may carry a passthrough somewhere inside it. +/// A tree that may carry a passthrough or a repeated map key somewhere +/// inside it: a source value ([`FfiValue`]) or a stored ciphertext +/// ([`StackCipherText`]), walked the one way the record rules need. trait RecordTree: Sized { /// The error a tree that does not fit its plan reports. const MISFIT: Error; - /// The tree as a row's entries, a batch's rows, or neither. - fn shape(self) -> Shape; - /// Whether this node is a passthrough. fn is_passthrough(&self) -> bool; @@ -689,16 +901,6 @@ trait RecordTree: Sized { fn children(&self) -> Children<'_, Self>; } -/// A tree read as rows. -enum Shape { - /// One row: its named entries. - Row(Vec<(String, T)>), - /// A batch: its rows, each still to be read as one. - Batch(Vec), - /// Neither. - Other, -} - /// A node's children. enum Children<'a, T> { Sequence(&'a [T]), @@ -709,14 +911,6 @@ enum Children<'a, T> { impl RecordTree for FfiValue { const MISFIT: Error = Error::Source; - fn shape(self) -> Shape { - match self { - FfiValue::Object(entries) => Shape::Row(entries), - FfiValue::Array(items) => Shape::Batch(items), - _ => Shape::Other, - } - } - fn is_passthrough(&self) -> bool { matches!(self, FfiValue::Passthrough(_)) } @@ -733,14 +927,6 @@ impl RecordTree for FfiValue { impl RecordTree for StackCipherText { const MISFIT: Error = Error::Record; - fn shape(self) -> Shape { - match self { - CipherText::Map(entries) => Shape::Row(entries), - CipherText::Sequence(items) => Shape::Batch(items), - _ => Shape::Other, - } - } - fn is_passthrough(&self) -> bool { matches!(self, CipherText::Passthrough(_)) } @@ -760,58 +946,11 @@ impl RecordTree for StackCipherText { } } -/// The rows of a call — one record, or a batch of them — carried with -/// whether they came as a batch, so the result takes the shape the input -/// had. -struct Rows { - rows: Vec, - batched: bool, -} - -/// Each row of `tree`, as its named entries: one row for a map, one per item -/// for a sequence of maps, and the tree's misfit error for anything else. -fn rows(tree: V) -> Result>, Error> { - match tree.shape() { - Shape::Row(entries) => Ok(Rows { - rows: vec![entries], - batched: false, - }), - Shape::Batch(items) => Ok(Rows { - rows: items - .into_iter() - .map(|item| match item.shape() { - Shape::Row(entries) => Ok(entries), - _ => Err(V::MISFIT), - }) - .collect::, Error>>()?, - batched: true, - }), - Shape::Other => Err(V::MISFIT), - } -} - -impl Rows { - fn try_map(self, f: impl FnMut(T) -> Result) -> Result, Error> { - Ok(Rows { - rows: self - .rows - .into_iter() - .map(f) - .collect::, Error>>()?, - batched: self.batched, - }) - } - - /// The rows in the shape the input had: `batch` over all of them for a - /// batch, the one row bare otherwise. - fn reshape(self, batch: impl FnOnce(Vec) -> T) -> Result { - let Rows { mut rows, batched } = self; - if batched { - Ok(batch(rows)) - } else { - rows.pop().ok_or(Error::Internal) - } - } +/// The rows of a call, as the engine's records: one, or a batch, so the +/// result takes the shape the input had. +enum Rows { + One(FieldValues), + Batch(Vec), } /// Take the one entry named `name` out of a row, whatever order the row had @@ -843,26 +982,16 @@ fn keys_are_unique(entries: &[(String, T)]) -> bool { /// repeated key itself — at seal, because a map it cannot open must never /// be produced, and at open, because a stale entry appended beside the /// current one *verifies* under the same per-entry AAD — but it does so -/// only once the value reaches it: on the encrypt side that is after the -/// plan check has passed, where a failure reads as this module's own bug, -/// and on the decrypt side after the row's keys have been requested. A +/// only once the value reaches it, after the plan check has passed. A /// `check_source`/`check_record` that let such a tree through would say /// "well-formed" of a value the operation then refuses, so the walk refuses /// it here, as the misfit it is. /// -/// On the passthrough half: on the encrypt side a source field value with one inside it must not -/// reach a `"c"` slot: a passthrough node is *unauthenticated by definition* -/// — on decrypt it hands its payload back with no AEAD opened — so admitting -/// one under a field the plan declares ciphertext-bearing would quietly -/// produce a slot whose bytes verify nothing. On the decrypt side a `"c"` -/// subtree with one inside it is the load-bearing half: `decrypt_as` -/// collects **zero** retrieve-requests for a passthrough and returns its -/// payload with no AEAD opened, so an attacker with write access to the -/// stored tree could replace a field's `"c"` subtree with a passthrough -/// carrying forged plaintext, and this check's absence would report it as a -/// successful decrypt. [`encrypt`] never produces a passthrough under `"c"`, -/// so the shape is unconditionally an error, and the encrypt-side check is -/// what makes that a round-trip invariant rather than data loss. +/// The passthrough half is the invariant the +/// [module docs](self#the-stored-record-is-wire-format) call load-bearing: +/// on the encrypt side a passthrough inside a sealed field's value would +/// produce a `"c"` subtree whose bytes verify nothing; on the decrypt side a +/// passthrough under `"c"` would be handed back as if it had been opened. fn check_tree(tree: &T) -> Result<(), Error> { if tree.is_passthrough() { return Err(T::MISFIT); @@ -880,29 +1009,43 @@ fn check_tree(tree: &T) -> Result<(), Error> { } // ============================================================================= -// Encrypt side +// Encrypt side: the source as the engine's record // ============================================================================= -/// The rows of a record source, each aligned to the plan's field order, with -/// everything that can be checked without a cipher checked: the source is -/// one object or an array of objects, every plan field is present exactly -/// once in every row and no row carries a field the plan does not name -/// (silently dropping a field on either side would lose data or index -/// nothing), and each value fits its field's outputs ([`check_field`]). -fn source_rows(source: FfiValue, plan: &Plan) -> Result>, Error> { - rows(source)?.try_map(|mut row| { - if row.len() != plan.fields.len() { - return Err(Error::Source); - } - plan.fields - .iter() - .map(|field| { - let (_, value) = take(&mut row, &field.name).ok_or(Error::Source)?; - check_field(&value, field)?; - Ok(value) - }) - .collect() - }) +/// The rows of a record source, each as the [`FieldValues`] the lowered +/// plan runs over, with everything that can be checked without a cipher +/// checked: the source is one object or an array of objects, every plan +/// field is present exactly once in every row and no row carries a field +/// the plan does not name, and each value fits its field ([`check_field`]). +/// Each field is moved out of the source into its slot at the type its kind +/// lowers to; nothing else is copied. +fn source_rows(source: FfiValue, plan: &Plan) -> Result { + match source { + FfiValue::Object(row) => Ok(Rows::One(source_row(row, plan)?)), + FfiValue::Array(items) => Ok(Rows::Batch( + items + .into_iter() + .map(|item| match item { + FfiValue::Object(row) => source_row(row, plan), + _ => Err(Error::Source), + }) + .collect::, _>>()?, + )), + _ => Err(Error::Source), + } +} + +fn source_row(mut row: Vec<(String, FfiValue)>, plan: &Plan) -> Result { + if row.len() != plan.fields.len() { + return Err(Error::Source); + } + let mut values = FieldValues::new(); + for field in &plan.fields { + let (_, value) = take(&mut row, &field.name).ok_or(Error::Source)?; + check_field(&value, field)?; + insert_leaf(&mut values, &field.name, field.leaf(), value, Error::Source)?; + } + Ok(values) } /// A source value against its plan field: a typed field needs a value of @@ -925,146 +1068,194 @@ fn check_field(value: &FfiValue, field: &FieldPlan) -> Result<(), Error> { return Err(Error::Term { kind: kind.clone() }); } } + Output::Passthrough => {} } } Ok(()) } -/// One field of a built row: its name and, per output in plan order, the -/// key and what fills it. -struct FieldSkeleton { - name: String, - outputs: Vec<(&'static str, Slot)>, +/// Put `value` in the record at the type `leaf` names. A value of another +/// kind than the leaf's — checked before this is reached on both paths — is +/// `misfit`. +fn insert_leaf( + values: &mut FieldValues, + name: &str, + leaf: Leaf, + value: FfiValue, + misfit: Error, +) -> Result<(), Error> { + let _ = match (leaf, value) { + (Leaf::U32, FfiValue::UInt32(v)) => values.insert(name, v), + (Leaf::Text, FfiValue::String(s)) => { + // Valid UTF-8 by `Utf8String`'s invariant; the bytes move, they + // are not copied. + let text = String::from_utf8(s.into_inner().risky_unwrap()).map_err(|_| misfit)?; + values.insert(name, text) + } + (Leaf::Value, value) => values.insert(name, Value::new(value)), + (Leaf::U32 | Leaf::Text, _) => return Err(misfit), + }; + Ok(()) } -/// What fills an output slot: a term, derived as the row was built, or the -/// ciphertext still pending in the row's batch, filled in build order once -/// the batch settles. -enum Slot { - Term(Vec), - Ciphertext, -} +// ============================================================================= +// Output adapters: the engine's record as the stored shape, and back +// ============================================================================= -/// The values a batch settled to, handed back one per slot in build order. -/// The count has to come out exact — a slot with no value, or a value with -/// no slot, means the merge miscounted, which is a bug here. -struct Settled(std::vec::IntoIter); +/// What the adapters know of a field once the engine has run. +#[derive(Clone, Debug)] +struct FieldShape { + name: String, + verb: Verb, + leaf: Leaf, + keys: Vec<&'static str>, + kind: Option, +} -impl Settled { - fn of(values: Vec) -> Self { - Self(values.into_iter()) - } +/// A term as the stored tree carries it: a passthrough byte node. +fn term_node(term: TermBytes) -> StackCipherText { + CipherText::Passthrough(Box::new(FfiValue::Bytes(vitaminc_protected::Protected::new( + term.into_bytes(), + ))) as BoxedPassthrough) +} - fn next(&mut self) -> Result { - self.0.next().ok_or(Error::Internal) +/// The record the plan produced, in the stored shape: per field, in plan +/// order, its output map. +fn shape_record( + mut values: FieldValues, + shape: &[FieldShape], +) -> Result { + let mut fields = Vec::with_capacity(shape.len()); + for field in shape { + let outputs = match field.verb { + Verb::Encrypt => { + let ciphertext: StackCipherText = values.take(&field.name)?; + vec![("c".to_string(), ciphertext)] + } + Verb::EncryptIndex => { + let sealed: Encrypted> = values.take(&field.name)?; + let mut outputs = Vec::with_capacity(1 + field.keys.len()); + outputs.push(("c".to_string(), sealed.ciphertext)); + outputs.extend(keyed_terms(sealed.terms, &field.keys)?); + outputs + } + Verb::Index => keyed_terms(values.take(&field.name)?, &field.keys)?, + Verb::Passthrough => { + let value = take_leaf(&mut values, &field.name, field.leaf)?; + vec![( + "passthrough".to_string(), + CipherText::Passthrough(Box::new(value) as BoxedPassthrough), + )] + } + }; + fields.push((field.name.clone(), CipherText::Map(outputs))); } + Ok(CipherText::Map(fields)) +} - fn finish(mut self) -> Result<(), Error> { - match self.0.next() { - Some(_) => Err(Error::Internal), - None => Ok(()), - } - } +/// Each term under its index key, in the order the plan named the indexes. +/// The counts cannot disagree: both come from the plan's indexes; if they +/// did, the engine answered with a different shape than it was asked. +fn keyed_terms( + terms: Vec, + keys: &[&'static str], +) -> Result, crate::Error> { + if terms.len() != keys.len() { + return Err(crate::Error::ResponseShape); + } + Ok(keys + .iter() + .zip(terms) + .map(|(key, term)| ((*key).to_string(), term_node(term))) + .collect()) } -/// Build one record row: derive its terms and queue its ciphertext pendings, -/// returning the row skeleton. The plan drives the iteration so the output -/// field order is the plan's; the row arrives from [`source_rows`] already -/// in that order and checked against the plan. -async fn build_row<'c, K>( - cipher: &'c KeysetCipher<'_, K>, - row: Vec, - plan: &Plan, - pendings: &mut Vec>, -) -> Result, Error> -where - K: DataKeySource + Sync, -{ - // A row `source_rows` did not align is a bug here, not caller input. - if row.len() != plan.fields.len() { - return Err(Error::Internal); - } - let mut skeleton = Vec::with_capacity(plan.fields.len()); - for (field, value) in plan.fields.iter().zip(row) { - let name = field.name.clone(); - // The one context this field has, cloned per output: this variable - // is what reaches the ciphertext and every term (ADR-0004), and - // there is no other. - let context = field.view()?; - - // Terms first — they lift a copy of the scalar; the value itself is - // consumed by the ciphertext path below. One lift serves every term - // output: the kind only names which error a non-scalar reports. - let scalar = field - .outputs - .iter() - .find_map(|o| match o { - Output::Term(kind) => Some(kind), - Output::Ciphertext => None, - }) - .map(|kind| Scalar::of(&value, kind)) - .transpose()?; - - let mut outputs = Vec::with_capacity(field.outputs.len()); - for output in &field.outputs { - let Output::Term(kind) = output else { - outputs.push((output.key(), Slot::Ciphertext)); - continue; - }; - let scalar = scalar.clone().ok_or(Error::Internal)?; - outputs.push(( - output.key(), - Slot::Term(term(cipher, scalar, kind, context.clone()).await?), - )); - } +/// The field `name` out of the record, as a value: the leaf type back to +/// the variant it came from. +fn take_leaf(values: &mut FieldValues, name: &str, leaf: Leaf) -> Result { + Ok(match leaf { + Leaf::U32 => FfiValue::UInt32(values.take(name)?), + Leaf::Text => FfiValue::String(values.take::(name)?.into()), + Leaf::Value => values.take::(name)?.into_inner(), + }) +} - if field.has_ciphertext() { - // Re-checked here so this function's own contract does not rest - // on its caller's: with the tree checked, the cipher's refusals - // (a passthrough, a repeated key) cannot fire, and a failure - // below is a bug here. - check_tree(&value)?; - let tree = value - .encrypt_with_aad(cipher, context.clone()) - .map_err(|_| Error::Internal)?; - pendings.push(tree.into_pending(cipher, context)); +/// The record the plan opened, as a value: the fields that come back, in +/// plan order, each checked against its declared kind. The tag is +/// authenticated, so a mismatch is not tampering: the row was sealed as +/// another type than the plan now declares. +fn open_record(mut values: FieldValues, shape: &[FieldShape]) -> Result { + let mut fields = Vec::with_capacity(shape.len()); + for field in shape.iter().filter(|field| field.verb != Verb::Index) { + let value = take_leaf(&mut values, &field.name, field.leaf)?; + if let Some(kind) = field.kind { + if !kind.holds(&value) { + return Err(crate::PlanError::FieldType { + field: field.name.clone(), + expected: kind.name(), + } + .into()); + } } - - skeleton.push(FieldSkeleton { name, outputs }); + fields.push((field.name.clone(), value)); } - Ok(skeleton) + Ok(FfiValue::Object(fields)) } // ============================================================================= -// Decrypt side +// Decrypt side: the stored tree as the engine's record // ============================================================================= -/// The `"c"` subtrees a record tree holds for the plan's ciphertext-bearing -/// fields, per row in plan order, with the row's field name: the tree is one -/// map or a sequence of maps, each such field is present exactly once, is a -/// map of outputs with exactly one `"c"` node, and that node has no -/// passthrough and no repeated key in it ([`check_tree`]). Terms and fields -/// the plan does not name are ignored (comparands, not ciphertext). -#[allow(clippy::type_complexity)] -fn record_leaves( - tree: StackCipherText, - plan: &Plan, -) -> Result>, Error> { - rows(tree)?.try_map(|mut row| { - plan.fields - .iter() - .filter(|field| field.has_ciphertext()) - .map(|field| { - let (name, node) = take(&mut row, &field.name).ok_or(Error::Record)?; - let Shape::Row(mut outputs) = node.shape() else { +/// The rows of a stored record, each as the [`FieldValues`] the lowered +/// plan opens: the tree is one map or a sequence of maps; each sealed field +/// is present exactly once, is a map of outputs with exactly one `"c"`, and +/// that node has no passthrough and no repeated key in it ([`check_tree`]); +/// each passthrough field is present exactly once with exactly one +/// `"passthrough"` node carrying a value of the field's kind. Terms, and +/// entries the plan does not open, are ignored: comparands, not ciphertext. +fn record_rows(tree: StackCipherText, plan: &Plan) -> Result { + match tree { + CipherText::Map(row) => Ok(Rows::One(record_row(row, plan)?)), + CipherText::Sequence(items) => Ok(Rows::Batch( + items + .into_iter() + .map(|item| match item { + CipherText::Map(row) => record_row(row, plan), + _ => Err(Error::Record), + }) + .collect::, _>>()?, + )), + _ => Err(Error::Record), + } +} + +fn record_row(mut row: Vec<(String, StackCipherText)>, plan: &Plan) -> Result { + let mut values = FieldValues::new(); + for field in plan.fields.iter().filter(|field| field.opens()) { + let (_, node) = take(&mut row, &field.name).ok_or(Error::Record)?; + let CipherText::Map(mut outputs) = node else { + return Err(Error::Record); + }; + match field.verb() { + Verb::Passthrough => { + let (_, node) = take(&mut outputs, "passthrough").ok_or(Error::Record)?; + let CipherText::Passthrough(payload) = node else { return Err(Error::Record); }; - let (_, ct) = take(&mut outputs, Output::Ciphertext.key()).ok_or(Error::Record)?; - check_tree(&ct)?; - Ok((name, ct)) - }) - .collect() - }) + let value = *payload.downcast::().map_err(|_| Error::Record)?; + if field.field_type.is_some_and(|kind| !kind.holds(&value)) { + return Err(Error::Record); + } + insert_leaf(&mut values, &field.name, field.leaf(), value, Error::Record)?; + } + Verb::Encrypt | Verb::EncryptIndex | Verb::Index => { + let (_, ciphertext) = take(&mut outputs, "c").ok_or(Error::Record)?; + check_tree(&ciphertext)?; + let _ = values.insert(&field.name, ciphertext); + } + } + } + Ok(values) } #[cfg(test)] @@ -1074,14 +1265,17 @@ mod tests { use std::sync::atomic::{AtomicUsize, Ordering}; use stack_kms::{ - DataKey, DataKeyWithTag, FakeDataKeySource, GenerateKeyPayload, IdentifiedBy, IndexKey, - IndexKeySource, RetrieveKeyPayload, UnverifiedContext, + DataKey, DataKeySource, DataKeyWithTag, FakeDataKeySource, GenerateKeyPayload, + IdentifiedBy, IndexKey, IndexKeySource, RetrieveKeyPayload, UnverifiedContext, }; use uuid::Uuid; - use vitaminc_protected::Controlled; + use vitaminc_protected::Protected; - use crate::dynamic::context; - use crate::{nonempty, StackCipher}; + use crate::dynamic::{context, term}; + use crate::plan::pick; + use crate::sem::{EqualityTerm, MatchTerms, OreTerm}; + use crate::target::{AeadContext, DecryptInto, EncryptFrom}; + use crate::{nonempty, Equality, Match, Ore, PlanError, StackCipher}; /// `FakeDataKeySource` with call counters, so the batching contract — /// one key request per invocation, none for a refused call — is @@ -1163,24 +1357,32 @@ mod tests { FfiValue::Array(items.iter().map(|item| s(item)).collect()) } + /// The label `users/`, as a binding spells a column. + fn label(field: &str) -> FfiValue { + strings(&["users", field]) + } + fn spec(context: FfiValue, outputs: &[&str]) -> FfiValue { obj(vec![("context", context), ("outputs", strings(outputs))]) } + fn typed(context: FfiValue, outputs: &[&str], ty: &str) -> FfiValue { + obj(vec![ + ("context", context), + ("outputs", strings(outputs)), + ("type", s(ty)), + ]) + } + /// The plan most tests share: `age` sealed and indexed for equality and - /// order under `"users/age"`; `email` sealed alone under an extended - /// context; `nick` indexed for match only, never sealed. + /// order under `users/age`; `email` sealed alone under `users/email`; + /// `nick` indexed for match only, never sealed; `id` carried through. fn plan_value() -> FfiValue { obj(vec![ - ("age", spec(s("users/age"), &["c", "eq", "ore"])), - ( - "email", - spec( - FfiValue::Array(vec![s("users/email"), FfiValue::UInt64(7)]), - &["c"], - ), - ), - ("nick", spec(s("users/nick"), &["match"])), + ("age", spec(label("age"), &["c", "eq", "ore"])), + ("email", spec(label("email"), &["c"])), + ("nick", spec(label("nick"), &["match"])), + ("id", spec(label("id"), &["passthrough"])), ]) } @@ -1193,9 +1395,34 @@ mod tests { ("age", FfiValue::UInt32(age)), ("email", s("a@x")), ("nick", s("al smith")), + ("id", FfiValue::UInt64(7)), ]) } + // ---- running the lowering --------------------------------------------- + + async fn seal( + keyset: &KeysetCipher<'_, Counting>, + source: FfiValue, + plan: &Plan, + ) -> StackCipherText { + encrypt(keyset, source, plan) + .expect("the source fits the plan") + .await + .expect("encrypt") + } + + async fn open( + cipher: &StackCipher, + record: StackCipherText, + plan: &Plan, + ) -> FfiValue { + decrypt(Scope::Client(cipher), record, plan) + .expect("the record fits the plan") + .await + .expect("decrypt") + } + // ---- reading results back ----------------------------------------------- fn map(tree: StackCipherText) -> Vec<(String, StackCipherText)> { @@ -1253,6 +1480,13 @@ mod tests { } } + fn u64_of(value: &FfiValue) -> u64 { + match value { + FfiValue::UInt64(v) => *v, + _ => panic!("expected a u64"), + } + } + fn text_of(value: &FfiValue) -> String { match value { FfiValue::String(s) => String::from_utf8(s.risky_ref().to_vec()).expect("utf8"), @@ -1264,21 +1498,11 @@ mod tests { CipherText::Passthrough(Box::new(value) as BoxedPassthrough) } - /// `Settled` is exact both ways: a slot with no value and a value with - /// no slot are both the merge miscounting, reported as `Internal`. - #[test] - fn settled_values_must_match_their_slots_exactly() { - let mut settled = Settled::of(vec![1]); - assert!(matches!(settled.next(), Ok(1))); - assert!( - matches!(settled.next(), Err(Error::Internal)), - "a slot with no value" - ); - assert!(Settled::of(Vec::::new()).finish().is_ok()); - assert!( - matches!(Settled::of(vec![1]).finish(), Err(Error::Internal)), - "a value with no slot" - ); + fn plan_error(error: crate::Error) -> PlanError { + match error { + crate::Error::Plan(error) => error, + other => panic!("expected a plan error, got {other:?}"), + } } /// A table row: what is refused, the value that must be refused, and @@ -1290,14 +1514,14 @@ mod tests { use super::*; #[test] - fn parses_each_field_in_order_with_its_context_and_outputs() { + fn parses_each_field_in_order_with_its_label_and_outputs() { let plan = the_plan(); assert_eq!( plan.fields() .iter() .map(FieldPlan::name) .collect::>(), - ["age", "email", "nick"], + ["age", "email", "nick", "id"], "fields keep the plan's order" ); assert_eq!( @@ -1321,16 +1545,29 @@ mod tests { ))], "a field can be indexed and never sealed" ); + assert_eq!( + plan.fields()[3].outputs(), + [Output::Passthrough], + "a field can be carried through" + ); assert!( plan.fields()[0].has_ciphertext() && plan.fields()[1].has_ciphertext() - && !plan.fields()[2].has_ciphertext(), + && !plan.fields()[2].has_ciphertext() + && !plan.fields()[3].has_ciphertext(), "has_ciphertext follows the outputs" ); + assert_eq!(plan.label().to_string(), "users", "the plan's one context"); + assert!(plan.extension().is_empty(), "no extension was spelled"); + assert_eq!( + plan.fields()[1].label().to_string(), + "users/email", + "a field's label is the one its spec spelled" + ); + assert_eq!(plan.fields()[1].identity(), "email"); assert_eq!( plan.fields()[1].context(), - &context(FfiValue::Array(vec![s("users/email"), FfiValue::UInt64(7)])) - .expect("context"), + &context(label("email")).expect("context"), "a field's context is the one its spec spelled, read by `context`" ); } @@ -1352,7 +1589,7 @@ mod tests { obj(vec![( "age", obj(vec![ - ("context", s("users/age")), + ("context", label("age")), ("outputs", strings(&["c"])), ("nullable", FfiValue::Bool(true)), ]), @@ -1366,12 +1603,12 @@ mod tests { ), ( "a field spec with no outputs", - obj(vec![("age", obj(vec![("context", s("users/age"))]))]), + obj(vec![("age", obj(vec![("context", label("age"))]))]), |e| matches!(e, Error::Plan), ), ( "outputs that are not a list", - obj(vec![("age", spec(s("users/age"), &[]))]), + obj(vec![("age", spec(label("age"), &[]))]), |e| matches!(e, Error::Plan), ), ( @@ -1379,7 +1616,7 @@ mod tests { obj(vec![( "age", obj(vec![ - ("context", s("users/age")), + ("context", label("age")), ("outputs", FfiValue::Array(vec![FfiValue::UInt32(1)])), ]), )]), @@ -1387,19 +1624,29 @@ mod tests { ), ( "an unknown output", - obj(vec![("age", spec(s("users/age"), &["c", "sum"]))]), + obj(vec![("age", spec(label("age"), &["c", "sum"]))]), |e| matches!(e, Error::Plan), ), ( "an output named twice", - obj(vec![("age", spec(s("users/age"), &["c", "eq", "c"]))]), + obj(vec![("age", spec(label("age"), &["c", "eq", "c"]))]), + |e| matches!(e, Error::Plan), + ), + ( + "passthrough beside a ciphertext", + obj(vec![("age", spec(label("age"), &["passthrough", "c"]))]), + |e| matches!(e, Error::Plan), + ), + ( + "passthrough beside an index", + obj(vec![("age", spec(label("age"), &["eq", "passthrough"]))]), |e| matches!(e, Error::Plan), ), ( "a field named twice", FfiValue::Object(vec![ - ("age".to_string(), spec(s("users/age"), &["c"])), - ("age".to_string(), spec(s("users/age"), &["eq"])), + ("age".to_string(), spec(label("age"), &["c"])), + ("age".to_string(), spec(label("age"), &["eq"])), ]), |e| matches!(e, Error::Plan), ), @@ -1408,9 +1655,9 @@ mod tests { obj(vec![( "age", obj(vec![ - ("context", s("users/age")), + ("context", label("age")), ("outputs", strings(&["c"])), - ("context", s("users/other")), + ("context", label("other")), ]), )]), |e| matches!(e, Error::Plan), @@ -1420,7 +1667,7 @@ mod tests { obj(vec![( "age", obj(vec![ - ("context", s("users/age")), + ("context", label("age")), ("outputs", strings(&["c"])), ("outputs", strings(&["eq"])), ]), @@ -1447,12 +1694,138 @@ mod tests { } } + /// A field's context is its label, optionally extended, and nothing + /// else: what a fields plan can give a field. The shapes refused here + /// are contexts (`dynamic::context` reads them) that no `.fields()` + /// chain could spell. #[test] - fn a_field_plan_refuses_no_outputs_and_a_repeated_output() { - let ctx = context(s("users/age")).expect("context"); + fn refuses_a_field_context_that_is_not_a_label() { + let refused = [ + ("one text part, however it reads", s("users/age")), + ("a one-segment label", strings(&["users"])), + ("a one-element list", FfiValue::Array(vec![s("users")])), + ("an integer", FfiValue::UInt64(7)), + ("bytes", FfiValue::Bytes(Protected::new(b"users".to_vec()))), + ( + "a one-segment label, extended", + FfiValue::Array(vec![s("users"), FfiValue::UInt64(7)]), + ), + ( + "a segment that is not plain", + strings(&["users", "age/years"]), + ), + ( + "a segment with a reserved prefix", + strings(&["users", "7age"]), + ), + ( + "a list part as an extension", + FfiValue::Array(vec![label("age"), strings(&["eu", "west"])]), + ), + ]; + for (what, context) in refused { + let result = plan(obj(vec![("age", spec(context, &["c"]))])); + assert!(matches!(result, Err(Error::Plan)), "{what}: {result:?}"); + } + } + + /// Every field sits under the plan's one context and carries the one + /// extension, so two fields that disagree are not one plan. + #[test] + fn refuses_fields_under_different_contexts_or_extensions() { + let parsed = plan(obj(vec![ + ("age", spec(label("age"), &["c"])), + ("total", spec(strings(&["orders", "total"]), &["c"])), + ])); assert!( - matches!(FieldPlan::new("age", ctx.clone(), vec![]), Err(Error::Plan)), - "a field must produce something" + matches!(parsed, Err(Error::Plan)), + "two contexts: {parsed:?}" + ); + let parsed = plan(obj(vec![ + ( + "age", + spec( + FfiValue::Array(vec![label("age"), FfiValue::UInt64(7)]), + &["c"], + ), + ), + ("email", spec(label("email"), &["c"])), + ])); + assert!( + matches!(parsed, Err(Error::Plan)), + "one field extended, one not: {parsed:?}" + ); + let parsed = plan(obj(vec![ + ( + "age", + spec( + FfiValue::Array(vec![label("age"), FfiValue::UInt64(7)]), + &["c"], + ), + ), + ( + "email", + spec( + FfiValue::Array(vec![label("email"), FfiValue::UInt64(8)]), + &["c"], + ), + ), + ])); + assert!( + matches!(parsed, Err(Error::Plan)), + "two extensions: {parsed:?}" + ); + // The same prefix spelled deeper is still one context. + let parsed = plan(obj(vec![ + ("age", spec(strings(&["app", "users", "age"]), &["c"])), + ("email", spec(strings(&["app", "users", "email"]), &["c"])), + ])) + .expect("one two-segment context"); + assert_eq!(parsed.label().to_string(), "app/users"); + } + + /// Two sealed fields keyed under one identity would have one context + /// and interchangeable terms: the builder refuses it, so the plan does. + #[test] + fn refuses_two_fields_keyed_under_one_identity() { + let parsed = plan(obj(vec![ + ("mail", spec(label("email"), &["c", "eq"])), + ("mail2", spec(label("email"), &["c", "eq"])), + ])); + assert!(matches!(parsed, Err(Error::Plan)), "{parsed:?}"); + // Two passthrough fields key nothing, so they may share a label. + let parsed = plan(obj(vec![ + ("a", spec(label("meta"), &["passthrough"])), + ("b", spec(label("meta"), &["passthrough"])), + ])); + assert!(parsed.is_ok(), "{parsed:?}"); + } + + /// A field whose label ends in a segment other than its name is keyed + /// under that segment: the plan pins its identity. + #[test] + fn a_label_whose_last_segment_is_not_the_name_pins_the_identity() { + let parsed = plan(obj(vec![( + "nickname", + spec(strings(&["users", "handle"]), &["c"]), + )])) + .expect("parses"); + assert_eq!(parsed.fields()[0].identity(), "handle"); + let lowered = parsed.lower::<()>().expect("lowers"); + let field = lowered.field("nickname").expect("the field"); + assert_eq!(field.identity(), "handle"); + assert_eq!( + field.label().map(ToString::to_string), + Some("users/handle".into()) + ); + } + + #[test] + fn a_field_plan_refuses_no_outputs_and_a_repeated_output() { + let ctx = context(label("age")).expect("context"); + assert!( + matches!(FieldPlan::new("age", ctx.clone(), vec![]), Err(Error::Plan)), + "a field must produce something" ); assert!( matches!( @@ -1465,6 +1838,17 @@ mod tests { ), "an output cannot be produced twice under one key" ); + assert!( + matches!( + FieldPlan::new( + "age", + ctx.clone(), + vec![Output::Passthrough, Output::Ciphertext] + ), + Err(Error::Plan) + ), + "a passthrough field has no other output" + ); assert!( FieldPlan::new("age", ctx, vec![Output::Ciphertext]).is_ok(), "one output is a plan" @@ -1488,7 +1872,7 @@ mod tests { let plan = plan(obj(vec![( "nick", obj(vec![ - ("context", s("users/nick")), + ("context", label("nick")), ("outputs", FfiValue::Array(vec![s("c"), wide])), ]), )])) @@ -1514,7 +1898,7 @@ mod tests { let parsed = plan(obj(vec![( "nick", obj(vec![ - ("context", s("users/nick")), + ("context", label("nick")), ("outputs", FfiValue::Array(vec![s("match"), wide])), ]), )])); @@ -1522,7 +1906,7 @@ mod tests { matches!(parsed, Err(Error::Plan)), "two match outputs are one key twice" ); - let ctx = context(s("users/nick")).expect("context"); + let ctx = context(label("nick")).expect("context"); let by_hand = FieldPlan::new( "nick", ctx, @@ -1537,11 +1921,12 @@ mod tests { assert!(matches!(by_hand, Err(Error::Plan)), "and by hand alike"); } - /// An output list entry is `"c"` or an index in its wire form, and - /// nothing else; a malformed match object is a plan error. + /// An output list entry is `"c"`, `"passthrough"` or an index in its + /// wire form, and nothing else; a malformed match object is a plan + /// error. #[test] fn an_output_that_is_not_one_is_refused() { - for (label, output) in [ + for (label_, output) in [ ("an unknown key", s("cc")), ("a number", FfiValue::UInt32(1)), ( @@ -1556,14 +1941,18 @@ mod tests { let parsed = plan(obj(vec![( "nick", obj(vec![ - ("context", s("users/nick")), + ("context", label("nick")), ("outputs", FfiValue::Array(vec![output])), ]), )])); - assert!(matches!(parsed, Err(Error::Plan)), "{label}"); + assert!(matches!(parsed, Err(Error::Plan)), "{label_}"); } assert_eq!(Output::parse("c"), Some(Output::Ciphertext)); + assert_eq!(Output::parse("passthrough"), Some(Output::Passthrough)); + assert_eq!(Output::parse("ore"), Some(Output::Term(IndexSpec::Ore))); assert_eq!(Output::parse("cc"), None); + assert_eq!(Output::Passthrough.key(), "passthrough"); + assert_eq!(Output::Ciphertext.key(), "c"); } /// The whole-plan rules hold for a plan built by hand, not only for @@ -1574,7 +1963,7 @@ mod tests { let field = |name: &str| { FieldPlan::new( name, - context(s("users/age")).expect("context"), + context(label(name)).expect("context"), vec![Output::Ciphertext], ) .expect("field") @@ -1605,7 +1994,7 @@ mod tests { mod given_a_source_that_does_not_fit_the_plan { use super::*; - /// `check_source` is the parser `encrypt` runs, so a binding's + /// `check_source` is the conversion `encrypt` runs, so a binding's /// boundary rejection and the operation's are the same error — and /// neither costs a key request. #[tokio::test] @@ -1627,7 +2016,11 @@ mod tests { ), ( "a row missing a plan field", - obj(vec![("age", FfiValue::UInt32(1)), ("email", s("a@x"))]), + obj(vec![ + ("age", FfiValue::UInt32(1)), + ("email", s("a@x")), + ("nick", s("al")), + ]), |e| matches!(e, Error::Source), ), ( @@ -1639,6 +2032,15 @@ mod tests { }, |e| matches!(e, Error::Source), ), + ( + "a row with a field the plan does not name in place of one it does", + { + let mut entries = object(row(1)); + entries[3].0 = "extra".to_string(); + FfiValue::Object(entries) + }, + |e| matches!(e, Error::Source), + ), ( "a passthrough under a sealed field", { @@ -1705,36 +2107,33 @@ mod tests { }, ), ]; - // A value is consumed by the call that checks it, so the table - // exercises the boundary parser and the operation is exercised - // below on the shapes a caller is likeliest to get wrong. - for (label, source, expected) in cases { + for (label_, source, expected) in cases { let err = check_source(source, &plan).err(); assert!( err.as_ref().is_some_and(expected), - "{label}: check_source must refuse it as the right error: {err:?}" + "{label_}: check_source must refuse it as the right error: {err:?}" ); } - let missing = obj(vec![("age", FfiValue::UInt32(1)), ("email", s("a@x"))]); - let err = encrypt(&keyset, missing, &plan).await.err(); + let missing = obj(vec![ + ("age", FfiValue::UInt32(1)), + ("email", s("a@x")), + ("nick", s("al")), + ]); + let err = encrypt(&keyset, missing, &plan).err(); assert!( matches!(err, Some(Error::Source)), "encrypt refuses a row missing a plan field: {err:?}" ); let mut entries = object(row(1)); entries[1].1 = FfiValue::Passthrough(Box::new(s("a@x"))); - let err = encrypt(&keyset, FfiValue::Object(entries), &plan) - .await - .err(); + let err = encrypt(&keyset, FfiValue::Object(entries), &plan).err(); assert!( matches!(err, Some(Error::Source)), "encrypt refuses a passthrough under a sealed field: {err:?}" ); let mut entries = object(row(1)); entries[2].1 = FfiValue::UInt32(3); - let err = encrypt(&keyset, FfiValue::Object(entries), &plan) - .await - .err(); + let err = encrypt(&keyset, FfiValue::Object(entries), &plan).err(); assert!( matches!( err, @@ -1756,9 +2155,9 @@ mod tests { let cipher = cipher().await; let keyset = cipher.default_keyset(); let plan = - plan(obj(vec![("score", spec(s("users/score"), &["c", "eq"]))])).expect("plan"); + plan(obj(vec![("score", spec(label("score"), &["c", "eq"]))])).expect("plan"); let source = obj(vec![("score", FfiValue::Float64(1.5))]); - let err = encrypt(&keyset, source, &plan).await.err(); + let err = encrypt(&keyset, source, &plan).err(); assert!( matches!( err, @@ -1776,12 +2175,18 @@ mod tests { use super::*; #[tokio::test] - async fn seals_it_from_one_key_request_in_the_plan_shape() { + async fn seals_it_from_one_key_request_in_the_stored_shape() { let cipher = cipher().await; let keyset = cipher.default_keyset(); let plan = the_plan(); - let sealed = encrypt(&keyset, row(34), &plan).await.expect("encrypt"); + let pending = encrypt(&keyset, row(34), &plan).expect("fits"); + assert_eq!( + generates(&cipher), + 0, + "nothing is requested before the await" + ); + let sealed = pending.await.expect("encrypt"); assert_eq!( generates(&cipher), 1, @@ -1791,14 +2196,14 @@ mod tests { let mut fields = map(sealed); assert_eq!( keys(&fields), - ["age", "email", "nick"], + ["age", "email", "nick", "id"], "the result holds every plan field, in plan order" ); let mut age = map(node(&mut fields, "age")); assert_eq!( keys(&age), ["c", "eq", "ore"], - "a field's outputs ride under their keys, in output order" + "a field's ciphertext rides first, then its terms in index order" ); assert!( !matches!(node(&mut age, "c"), CipherText::Passthrough(_)), @@ -1821,28 +2226,62 @@ mod tests { ["match"], "an indexed-only field has just its term" ); + let mut id = map(node(&mut fields, "id")); + assert_eq!(keys(&id), ["passthrough"]); + let CipherText::Passthrough(payload) = node(&mut id, "passthrough") else { + panic!("a passthrough field rides as a passthrough node"); + }; + assert_eq!( + u64_of(payload.downcast_ref::().expect("a value")), + 7, + "carrying the value as it is" + ); + } + + /// The terms a plan lists after the ciphertext ride in the order the + /// plan named them, whichever order the spec spelled `"c"` in. + #[tokio::test] + async fn terms_ride_in_index_order_after_the_ciphertext() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let plan = + plan(obj(vec![("age", spec(label("age"), &["ore", "c", "eq"]))])).expect("plan"); + let sealed = seal(&keyset, obj(vec![("age", FfiValue::UInt32(1))]), &plan).await; + let mut fields = map(sealed); + let age = map(node(&mut fields, "age")); + assert_eq!(keys(&age), ["c", "ore", "eq"]); } /// ADR-0004's property, pinned: the ciphertext opens under the plan - /// context and under nothing else, and each term is the standalone - /// derivation under that same context. + /// label and under nothing else, and each term is the standalone + /// derivation under that same label. #[tokio::test] - async fn binds_the_ciphertext_and_every_term_under_the_one_plan_context() { + async fn binds_the_ciphertext_and_every_term_under_the_one_field_label() { let cipher = cipher().await; let keyset = cipher.default_keyset(); let plan = the_plan(); let age_ctx = plan.fields()[0].context().clone(); let nick_ctx = plan.fields()[2].context().clone(); - let mut fields = map(encrypt(&keyset, row(34), &plan).await.expect("encrypt")); + let mut fields = map(seal(&keyset, row(34), &plan).await); let mut age = map(node(&mut fields, "age")); let mut nick = map(node(&mut fields, "nick")); let opened: FfiValue = cipher .decrypt(node(&mut age, "c"), age_ctx.clone()) .await - .expect("the ciphertext opens under the plan context"); + .expect("the ciphertext opens under the field's label"); assert_eq!(u32_of(&opened), 34, "and to the value that was sealed"); + let mut again = map(seal(&keyset, row(34), &plan).await); + let mut again_age = map(node(&mut again, "age")); + let under_label: FfiValue = cipher + .decrypt( + node(&mut again_age, "c"), + Label::parse("users/age").expect("label"), + ) + .await + .expect("the field's context is the typed label"); + assert_eq!(u32_of(&under_label), 34); let mut email = map(node(&mut fields, "email")); let wrong: Result = @@ -1863,7 +2302,7 @@ mod tests { assert_eq!( term_bytes(&node(&mut age, "eq")), eq, - "the equality term is the standalone derivation under the plan context" + "the equality term is the standalone derivation under the field label" ); let ore = term(&keyset, Scalar::U32(34), &IndexSpec::Ore, age_ctx) .await @@ -1871,7 +2310,7 @@ mod tests { assert_eq!( term_bytes(&node(&mut age, "ore")), ore, - "the ore term is the standalone derivation under the plan context" + "the ore term is the standalone derivation under the field label" ); let scalar = Scalar::of( &s("al smith"), @@ -1889,7 +2328,7 @@ mod tests { assert_eq!( term_bytes(&node(&mut nick, "match")), matched, - "the match term is the standalone derivation under the plan context" + "the match term is the standalone derivation under the field label" ); } @@ -1907,7 +2346,7 @@ mod tests { let plan = plan(obj(vec![( "nick", obj(vec![ - ("context", s("users/nick")), + ("context", label("nick")), ( "outputs", FfiValue::Array(vec![IndexSpec::Match(options.clone()).to_value()]), @@ -1917,7 +2356,7 @@ mod tests { .expect("parses"); let ctx = plan.fields()[0].context().clone(); let row = obj(vec![("nick", s("al smith"))]); - let mut fields = map(encrypt(&keyset, row, &plan).await.expect("encrypt")); + let mut fields = map(seal(&keyset, row, &plan).await); let mut nick = map(node(&mut fields, "nick")); let stored = term_bytes(&node(&mut nick, "match")); @@ -1933,15 +2372,19 @@ mod tests { } #[tokio::test] - async fn opens_back_to_its_ciphertext_bearing_fields_in_plan_order() { + async fn opens_back_to_the_fields_that_come_back_in_plan_order() { let cipher = cipher().await; let keyset = cipher.default_keyset(); let plan = the_plan(); - let sealed = encrypt(&keyset, row(34), &plan).await.expect("encrypt"); + let sealed = seal(&keyset, row(34), &plan).await; - let opened = decrypt(Scope::Client(&cipher), sealed, &plan) - .await - .expect("decrypt"); + let pending = decrypt(Scope::Client(&cipher), sealed, &plan).expect("fits"); + assert_eq!( + retrieves(&cipher), + 0, + "nothing is retrieved before the await" + ); + let opened = pending.await.expect("decrypt"); assert_eq!( retrieves(&cipher), 1, @@ -1950,16 +2393,18 @@ mod tests { let fields = object(opened); assert_eq!( keys(&fields), - ["age", "email"], - "only the sealed fields come back, in plan order; terms are one-way" + ["age", "email", "id"], + "sealed and passthrough fields come back, in plan order; terms are one-way" ); assert_eq!(u32_of(&fields[0].1), 34, "the age round-trips"); assert_eq!(text_of(&fields[1].1), "a@x", "the email round-trips"); + assert_eq!(u64_of(&fields[2].1), 7, "the passthrough round-trips"); // Through the keyset it was sealed under, too. - let sealed = encrypt(&keyset, row(35), &plan).await.expect("encrypt"); + let sealed = seal(&keyset, row(35), &plan).await; let fields = object( decrypt(Scope::Keyset(keyset.clone()), sealed, &plan) + .expect("fits") .await .expect("decrypt through the keyset"), ); @@ -1985,20 +2430,277 @@ mod tests { }; check_source(source(), &plan).expect("check_source accepts it"); - let sealed = encrypt(&keyset, source(), &plan).await.expect("encrypt"); + let sealed = seal(&keyset, source(), &plan).await; check_record(sealed, &plan).expect("check_record accepts it"); - let sealed = encrypt(&keyset, source(), &plan).await.expect("encrypt"); - let fields = object( - decrypt(Scope::Client(&cipher), sealed, &plan) - .await - .expect("decrypt"), - ); + let sealed = seal(&keyset, source(), &plan).await; + let fields = object(open(&cipher, sealed, &plan).await); let email = object(fields.into_iter().nth(1).expect("the email field").1); assert_eq!(keys(&email), ["home", "work"]); assert_eq!(text_of(&email[0].1), "a@x"); assert_eq!(text_of(&email[1].1), "b@x"); } + + /// The lowering is the plan builder: the record a data plan seals is + /// the record the typed chain seals under the same declaration, field + /// for field — the same terms, and ciphertexts each side opens. + #[tokio::test] + async fn seals_what_the_typed_chain_seals_under_the_same_declaration() { + struct User { + age: u32, + email: String, + nick: String, + id: u64, + } + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let user = User { + age: 34, + email: "a@x".into(), + nick: "al smith".into(), + id: 7, + }; + let mut typed = cipher + .encrypt(&user) + .context("users") + .fields() + .encrypt_index(pick("age", |u: &User| &u.age), (Equality, Ore)) + .encrypt(pick("email", |u: &User| &u.email)) + .index(pick("nick", |u: &User| &u.nick), Match::default()) + .passthrough(pick("id", |u: &User| &u.id)) + .await + .expect("the typed chain"); + let plan = plan(obj(vec![ + ( + "age", + typed_spec(label("age"), &["c", "eq", "ore"], "uint32"), + ), + ("email", typed_spec(label("email"), &["c"], "string")), + ("nick", typed_spec(label("nick"), &["match"], "string")), + ("id", spec(label("id"), &["passthrough"])), + ])) + .expect("plan"); + let mut fields = map(seal(&keyset, row(34), &plan).await); + + let age: Encrypted<(EqualityTerm, OreTerm)> = typed.take("age").expect("age"); + let mut lowered_age = map(node(&mut fields, "age")); + assert_eq!( + term_bytes(&node(&mut lowered_age, "eq")), + age.terms.0.to_bytes(), + "the same equality term" + ); + assert_eq!( + term_bytes(&node(&mut lowered_age, "ore")), + age.terms.1.to_bytes(), + "the same ore term" + ); + let nick: MatchTerms = typed.take("nick").expect("nick"); + let mut lowered_nick = map(node(&mut fields, "nick")); + assert_eq!( + term_bytes(&node(&mut lowered_nick, "match")), + nick.to_bytes(), + "the same match terms" + ); + + // Each side's ciphertext opens through the other. + let typed_opened: u32 = cipher + .decrypt( + node(&mut lowered_age, "c"), + Label::parse("users/age").expect("label"), + ) + .await + .expect("the typed reader opens the lowering's u32"); + assert_eq!(typed_opened, 34); + let email: StackCipherText = typed.take("email").expect("email"); + let mut lowered_email = map(node(&mut fields, "email")); + let typed_email: String = cipher + .decrypt( + node(&mut lowered_email, "c"), + Label::parse("users/email").expect("label"), + ) + .await + .expect("the typed reader opens the lowering's string"); + assert_eq!(typed_email, "a@x"); + let stored = CipherText::Map(vec![ + ( + "age".to_string(), + CipherText::Map(vec![("c".to_string(), age.ciphertext)]), + ), + ( + "email".to_string(), + CipherText::Map(vec![("c".to_string(), email)]), + ), + ( + "id".to_string(), + CipherText::Map(vec![( + "passthrough".to_string(), + forged(FfiValue::UInt64(7)), + )]), + ), + ]); + let opened = object(open(&cipher, stored, &plan).await); + assert_eq!(keys(&opened), ["age", "email", "id"]); + assert_eq!( + u32_of(&opened[0].1), + 34, + "the lowering opens the typed chain's u32" + ); + assert_eq!(text_of(&opened[1].1), "a@x", "and its string"); + assert_eq!(u64_of(&opened[2].1), 7); + } + } + + fn typed_spec(context: FfiValue, outputs: &[&str], ty: &str) -> FfiValue { + typed(context, outputs, ty) + } + + mod given_an_extended_context { + use super::*; + + fn extended(field: &str, parts: &[FfiValue]) -> FfiValue { + parts.iter().fold(label(field), |context, part| { + FfiValue::Array(vec![context, duplicate_scalar(part)]) + }) + } + + fn duplicate_scalar(part: &FfiValue) -> FfiValue { + match part { + FfiValue::UInt64(v) => FfiValue::UInt64(*v), + FfiValue::String(t) => { + FfiValue::String(std::str::from_utf8(t.risky_ref()).expect("utf8").into()) + } + _ => panic!("a scalar part"), + } + } + + /// A field's extension lowers to the chain's `.extend(..)`: the + /// record is the typed chain's under the same extension, and the + /// stored terms are the probes under the extended context, not the + /// flat one. + #[tokio::test] + async fn a_one_part_extension_is_the_chains_extend() { + struct Age { + age: u32, + } + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let parts = [FfiValue::UInt64(7)]; + let plan = plan(obj(vec![( + "age", + typed(extended("age", &parts), &["c", "eq"], "uint32"), + )])) + .expect("plan"); + assert_eq!(plan.extension().len(), 1); + let mut fields = + map(seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &plan).await); + let mut age = map(node(&mut fields, "age")); + + let mut typed_record = cipher + .encrypt(&Age { age: 34 }) + .context("users") + .fields() + .encrypt_index(pick("age", |a: &Age| &a.age), Equality) + .extend(7u64) + .await + .expect("typed chain"); + let typed_age: Encrypted = typed_record.take("age").expect("age"); + let stored = term_bytes(&node(&mut age, "eq")); + assert_eq!( + stored, + typed_age.terms.to_bytes(), + "the same term as the chain extended by the same part" + ); + let native = nonempty!("users").with("age").with(7u64); + let probe = keyset.equality_term(34u32, native).await.expect("probe"); + assert_eq!(stored, probe.to_bytes()); + let flat = keyset + .equality_term(34u32, nonempty!("users").with("age")) + .await + .expect("probe"); + assert_ne!( + stored, + flat.to_bytes(), + "the extension domain-separates from the flat label" + ); + let opened: u32 = cipher + .decrypt(node(&mut age, "c"), native) + .await + .expect("the leaf opens under the extended context"); + assert_eq!(opened, 34); + } + + /// Several parts nest to the left, one at a time, as a binding + /// extends them and as `NonEmpty::with` nests: `((label)/7)/eu`, + /// never `(label)/(7/eu)`. + #[tokio::test] + async fn several_parts_nest_to_the_left() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let parts = [FfiValue::UInt64(7), s("eu")]; + let plan = plan(obj(vec![( + "age", + spec(extended("age", &parts), &["c", "eq"]), + )])) + .expect("plan"); + assert_eq!(plan.extension().len(), 2); + let mut fields = + map(seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &plan).await); + let mut age = map(node(&mut fields, "age")); + let stored = term_bytes(&node(&mut age, "eq")); + + let nested = nonempty!("users").with("age").with(7u64).with("eu"); + let probe = keyset.equality_term(34u32, nested).await.expect("probe"); + assert_eq!(stored, probe.to_bytes(), "left-nested, part by part"); + let one_piece = nonempty!("users") + .with("age") + .with(NonEmpty::from(7u64).with("eu")); + let probe = keyset.equality_term(34u32, one_piece).await.expect("probe"); + assert_ne!(stored, probe.to_bytes(), "not one two-part piece"); + + let opened: FfiValue = cipher + .decrypt(node(&mut age, "c"), nested) + .await + .expect("opens under the nested context"); + assert_eq!(u32_of(&opened), 34); + + // And the record opens through the plan, which carries the parts. + let sealed = seal(&keyset, obj(vec![("age", FfiValue::UInt32(35))]), &plan).await; + let opened = object(open(&cipher, sealed, &plan).await); + assert_eq!(u32_of(&opened[0].1), 35); + } + + /// `DeclaredContext::with` is what the lowering builds: one part is + /// the context a `NonEmpty` extension always gave, and each further + /// part nests to the left. + #[test] + fn declared_context_with_nests_like_nonempty_with() { + use crate::IntoAad; + let own = || nonempty!("users").with("age"); + let one = DeclaredContext::from(7u64).under(own()); + let with = DeclaredContext::default() + .with(CallerContext::from(7u64)) + .under(own()); + assert_eq!( + one.clone().into_aad().as_bytes(), + with.into_aad().as_bytes(), + "one part: `with` is `From`" + ); + assert_eq!( + one.into_aad().as_bytes(), + own().with(7u64).into_aad().as_bytes() + ); + let two = DeclaredContext::default() + .with(CallerContext::from(7u64)) + .with(CallerContext::from(nonempty!("eu"))) + .under(own()); + assert_eq!( + two.into_aad().as_bytes(), + own().with(7u64).with("eu").into_aad().as_bytes(), + "two parts nest to the left" + ); + let none = DeclaredContext::default().under(own()); + assert_eq!(none.into_aad().as_bytes(), own().into_aad().as_bytes()); + } } mod given_a_batch { @@ -2010,20 +2712,17 @@ mod tests { let keyset = cipher.default_keyset(); let plan = the_plan(); - let sealed = encrypt( + let sealed = seal( &keyset, FfiValue::Array(vec![row(1), row(2), row(3)]), &plan, ) - .await - .expect("encrypt"); + .await; assert_eq!(generates(&cipher), 1, "one key request for the whole batch"); let rows = sequence(sealed); assert_eq!(rows.len(), 3, "a batch seals to a sequence of rows"); - let opened = decrypt(Scope::Client(&cipher), CipherText::Sequence(rows), &plan) - .await - .expect("decrypt"); + let opened = open(&cipher, CipherText::Sequence(rows), &plan).await; assert_eq!( retrieves(&cipher), 1, @@ -2044,16 +2743,12 @@ mod tests { let cipher = cipher().await; let keyset = cipher.default_keyset(); let plan = the_plan(); - let sealed = encrypt(&keyset, FfiValue::Array(vec![]), &plan) - .await - .expect("an empty batch seals"); + let sealed = seal(&keyset, FfiValue::Array(vec![]), &plan).await; assert!( matches!(&sealed, CipherText::Sequence(rows) if rows.is_empty()), "an empty batch seals to an empty sequence" ); - let opened = decrypt(Scope::Client(&cipher), sealed, &plan) - .await - .expect("an empty batch opens"); + let opened = open(&cipher, sealed, &plan).await; assert!( array(opened).is_empty(), "an empty sequence opens to an empty array" @@ -2070,14 +2765,12 @@ mod tests { use super::*; async fn sealed(keyset: &KeysetCipher<'_, Counting>) -> Vec<(String, StackCipherText)> { - map(encrypt(keyset, row(34), &the_plan()) - .await - .expect("encrypt")) + map(seal(keyset, row(34), &the_plan()).await) } /// The forged-plaintext case the module docs call load-bearing is /// in here: a passthrough under `"c"` must be refused, because - /// `decrypt_as` would otherwise hand its payload back as if opened. + /// opening it would hand its payload back as if opened. #[tokio::test] async fn decrypt_and_check_record_refuse_it_before_any_key_is_retrieved() { let cipher = cipher().await; @@ -2101,6 +2794,13 @@ mod tests { let _ = node(&mut fields, "email"); cases.push(("a record missing a sealed field", CipherText::Map(fields))); + let mut fields = sealed(&keyset).await; + let _ = node(&mut fields, "id"); + cases.push(( + "a record missing a passthrough field", + CipherText::Map(fields), + )); + let mut fields = sealed(&keyset).await; let mut age = map(node(&mut fields, "age")); fields.push(("age".to_string(), node(&mut age, "c"))); @@ -2141,6 +2841,27 @@ mod tests { CipherText::Map(fields), )); + let mut fields = sealed(&keyset).await; + let mut id = map(node(&mut fields, "id")); + let _ = node(&mut id, "passthrough"); + fields.push(("id".to_string(), CipherText::Map(id))); + cases.push(( + "a passthrough field with no passthrough output", + CipherText::Map(fields), + )); + + let mut fields = sealed(&keyset).await; + let mut id = map(node(&mut fields, "id")); + let _ = node(&mut id, "passthrough"); + let mut stale = sealed(&keyset).await; + let mut stale_age = map(node(&mut stale, "age")); + id.push(("passthrough".to_string(), node(&mut stale_age, "c"))); + fields.push(("id".to_string(), CipherText::Map(id))); + cases.push(( + "a passthrough field carrying a ciphertext", + CipherText::Map(fields), + )); + // The three shapes where a first-match take would have picked // one of two valid ciphertexts: a second, stale-but-valid copy // of a field, of its `"c"` output, or of a key inside it. @@ -2173,11 +2894,11 @@ mod tests { )); let before = retrieves(&cipher); - for (label, record) in cases { - let err = decrypt(Scope::Client(&cipher), record, &plan).await.err(); + for (label_, record) in cases { + let err = decrypt(Scope::Client(&cipher), record, &plan).err(); assert!( matches!(err, Some(Error::Record)), - "{label}: decrypt must refuse it as a misfit record: {err:?}" + "{label_}: decrypt must refuse it as a misfit record: {err:?}" ); } assert_eq!( @@ -2227,12 +2948,12 @@ mod tests { fields.push(("age".to_string(), CipherText::Map(age))); fields.push(("extra".to_string(), forged(s("not a field")))); - let opened = object( - decrypt(Scope::Client(&cipher), CipherText::Map(fields), &plan) - .await - .expect("decrypt"), + let opened = object(open(&cipher, CipherText::Map(fields), &plan).await); + assert_eq!( + keys(&opened), + ["age", "email", "id"], + "the fields that come back open" ); - assert_eq!(keys(&opened), ["age", "email"], "the sealed fields open"); assert_eq!(u32_of(&opened[0].1), 34, "to what was sealed"); } } @@ -2251,14 +2972,15 @@ mod tests { let globex = cipher.keyset(named("globex")).await.expect("globex"); let plan = the_plan(); - let sealed = encrypt(&acme, row(34), &plan).await.expect("encrypt"); + let sealed = seal(&acme, row(34), &plan).await; let err = decrypt(Scope::Keyset(globex.clone()), sealed, &plan) + .expect("the record fits") .await .err(); assert!( matches!( err, - Some(Error::Cipher(crate::Error::ForeignKeyset { expected, found })) + Some(crate::Error::ForeignKeyset { expected, found }) if expected == globex.keyset_id() && found == acme.keyset_id() ), "another tenant's keyset refuses the leaf, naming both keysets: {err:?}" @@ -2269,21 +2991,22 @@ mod tests { "refused before any key was retrieved" ); - let sealed = encrypt(&acme, row(34), &plan).await.expect("encrypt"); + let sealed = seal(&acme, row(34), &plan).await; let opened = object( decrypt(Scope::Keyset(acme.clone()), sealed, &plan) + .expect("fits") .await .expect("its own keyset opens it"), ); assert_eq!(u32_of(&opened[0].1), 34, "to what was sealed"); - let sealed = encrypt(&acme, row(34), &plan).await.expect("encrypt"); - let opened = object( - decrypt(Scope::Client(&cipher), sealed, &plan) - .await - .expect("the client opens a leaf from any of its keysets"), + let sealed = seal(&acme, row(34), &plan).await; + let opened = object(open(&cipher, sealed, &plan).await); + assert_eq!( + u32_of(&opened[0].1), + 34, + "the client opens a leaf from any of its keysets" ); - assert_eq!(u32_of(&opened[0].1), 34, "to what was sealed"); } #[tokio::test] @@ -2300,11 +3023,11 @@ mod tests { } } - /// The typed helper the tests lean on, pinned in passing: `nonempty!` - /// and `context` agree, so a test written against either is the same - /// test. + /// The typed helper the tests lean on, pinned in passing: a field's + /// list context and the typed label agree, so a test written against + /// either is the same test. #[test] - fn the_plan_context_is_the_typed_context() { + fn the_field_context_is_the_typed_label() { use crate::IntoAad; assert_eq!( the_plan().fields()[0] @@ -2313,26 +3036,21 @@ mod tests { .into_inner() .into_aad() .as_bytes(), - nonempty!("users/age").into_aad().as_bytes(), - "a bare plan string is the typed literal" + Label::parse("users/age") + .expect("label") + .into_aad() + .as_bytes(), + "a field's list context is the label" ); } mod given_a_typed_field { use super::*; - fn typed(context: &str, outputs: &[&str], ty: &str) -> FfiValue { - obj(vec![ - ("context", s(context)), - ("outputs", strings(outputs)), - ("type", s(ty)), - ]) - } - fn age_plan(ty: &str) -> Plan { plan(obj(vec![( "age", - typed("users/age", &["c", "eq", "ore"], ty), + typed(label("age"), &["c", "eq", "ore"], ty), )])) .expect("a typed plan parses") } @@ -2340,9 +3058,9 @@ mod tests { #[test] fn the_plan_parses_the_type_and_an_untyped_field_has_none() { let parsed = plan(obj(vec![ - ("age", typed("users/age", &["c", "ore"], "uint64")), - ("bio", typed("users/bio", &["c", "match"], "string")), - ("notes", spec(s("users/notes"), &["c"])), + ("age", typed(label("age"), &["c", "ore"], "uint64")), + ("bio", typed(label("bio"), &["c", "match"], "string")), + ("notes", spec(label("notes"), &["c"])), ])) .expect("parses"); let types: Vec<_> = parsed.fields().iter().map(FieldPlan::field_type).collect(); @@ -2362,7 +3080,7 @@ mod tests { obj(vec![ ("type", s(ty)), ("outputs", strings(&["c", "eq"])), - ("context", s("users/age")), + ("context", label("age")), ]) }; let parsed = plan(obj(vec![("age", early("uint64"))])).expect("parses"); @@ -2381,15 +3099,15 @@ mod tests { #[test] fn the_plan_refuses_an_unresolvable_type_or_one_that_does_not_admit_an_index() { let refused: [(&str, FfiValue); 8] = [ - ("an unknown type", typed("users/x", &["c"], "u64")), + ("an unknown type", typed(label("x"), &["c"], "u64")), ( "a type in the wrong case", - typed("users/x", &["c"], "UInt64"), + typed(label("x"), &["c"], "UInt64"), ), ( "a type that is not a string", obj(vec![ - ("context", s("users/x")), + ("context", label("x")), ("outputs", strings(&["c"])), ("type", FfiValue::UInt32(6)), ]), @@ -2397,7 +3115,7 @@ mod tests { ( "a type given twice", obj(vec![ - ("context", s("users/x")), + ("context", label("x")), ("outputs", strings(&["c"])), ("type", s("string")), ("type", s("string")), @@ -2405,21 +3123,21 @@ mod tests { ), ( "match on an integer", - typed("users/x", &["c", "match"], "int64"), + typed(label("x"), &["c", "match"], "int64"), ), ( "equality on a float", - typed("users/x", &["c", "eq"], "float64"), + typed(label("x"), &["c", "eq"], "float64"), ), - ("equality on a bool", typed("users/x", &["eq"], "bool")), + ("equality on a bool", typed(label("x"), &["eq"], "bool")), ( "order on a composite", - typed("users/x", &["c", "ore"], "object"), + typed(label("x"), &["c", "ore"], "object"), ), ]; - for (label, field) in refused { + for (label_, field) in refused { let result = plan(obj(vec![("x", field)])); - assert!(matches!(result, Err(Error::Plan)), "{label}: {result:?}"); + assert!(matches!(result, Err(Error::Plan)), "{label_}: {result:?}"); } } @@ -2427,7 +3145,7 @@ mod tests { fn a_hand_built_field_takes_a_type_its_indexes_admit_and_refuses_one_they_do_not() { let field = FieldPlan::new( "age", - context(s("users/age")).expect("context"), + context(label("age")).expect("context"), vec![Output::Ciphertext, Output::Term(IndexSpec::Equality)], ) .expect("field"); @@ -2443,7 +3161,7 @@ mod tests { )); let sealed_only = FieldPlan::new( "doc", - context(s("users/doc")).expect("context"), + context(label("doc")).expect("context"), vec![Output::Ciphertext], ) .expect("field") @@ -2454,25 +3172,47 @@ mod tests { /// The engine verifies the tag rather than trusting the binding: a /// `u32` is not a `uint64`, however small. Refused before any key - /// is minted, by `check_source` alike. + /// is minted, by `check_source` alike — and for the two kinds that + /// lower to a Rust leaf type as well. #[tokio::test] async fn encrypt_refuses_a_value_of_another_type_with_no_key_request() { let cipher = cipher().await; let keyset = cipher.default_keyset(); - let plan = age_plan("uint64"); - let check = check_source(obj(vec![("age", FfiValue::UInt32(34))]), &plan); + let u64_plan = age_plan("uint64"); + let check = check_source(obj(vec![("age", FfiValue::UInt32(34))]), &u64_plan); assert!( matches!(check, Err(Error::Source)), "check_source refuses it too" ); - for (label, value) in [ + for (label_, value) in [ ("a u32 for a uint64 field", FfiValue::UInt32(34)), ("a float for a uint64 field", FfiValue::Float64(34.0)), ("null for a uint64 field", FfiValue::Null), ] { - let result = encrypt(&keyset, obj(vec![("age", value)]), &plan).await; - assert!(matches!(result, Err(Error::Source)), "{label}: {result:?}"); + let result = encrypt(&keyset, obj(vec![("age", value)]), &u64_plan).err(); + assert!( + matches!(result, Some(Error::Source)), + "{label_}: {result:?}" + ); } + let as_u32 = age_plan("uint32"); + let result = encrypt(&keyset, obj(vec![("age", FfiValue::UInt64(34))]), &as_u32).err(); + assert!( + matches!(result, Some(Error::Source)), + "a u64 for a uint32 field: {result:?}" + ); + let as_string = + plan(obj(vec![("age", typed(label("age"), &["c"], "string"))])).expect("plan"); + let result = encrypt( + &keyset, + obj(vec![("age", FfiValue::UInt32(34))]), + &as_string, + ) + .err(); + assert!( + matches!(result, Some(Error::Source)), + "a u32 for a string field: {result:?}" + ); assert_eq!(generates(&cipher), 0, "refused before any key request"); } @@ -2481,24 +3221,18 @@ mod tests { let cipher = cipher().await; let keyset = cipher.default_keyset(); let plan = age_plan("uint64"); - let sealed = encrypt(&keyset, obj(vec![("age", FfiValue::UInt64(34))]), &plan) - .await - .expect("seal"); + let sealed = seal(&keyset, obj(vec![("age", FfiValue::UInt64(34))]), &plan).await; let mut row = map(sealed); let mut age = map(node(&mut row, "age")); let eq = term_bytes(&node(&mut age, "eq")); let typed = keyset - .equality_term(34u64, nonempty!("users/age")) + .equality_term(34u64, nonempty!("users").with("age")) .await .expect("typed"); assert_eq!(eq, typed.into_bytes().to_vec(), "the u64 term"); - let sealed = encrypt(&keyset, obj(vec![("age", FfiValue::UInt64(34))]), &plan) - .await - .expect("seal"); - let opened = decrypt(Scope::Client(&cipher), sealed, &plan) - .await - .expect("open"); + let sealed = seal(&keyset, obj(vec![("age", FfiValue::UInt64(34))]), &plan).await; + let opened = open(&cipher, sealed, &plan).await; let fields = object(opened); assert!(matches!(&fields[..], [(name, FfiValue::UInt64(34))] if name == "age")); } @@ -2506,29 +3240,32 @@ mod tests { /// A row sealed as one type and read under a plan declaring another /// is refused, not handed back as the wrong type. The tag is inside /// the AEAD envelope, so this is a plan disagreeing with its data, - /// never tampering, and it is caught once the leaf is open. + /// never tampering, and it is caught once the leaf is open — which + /// is the pending's failure, not the preflight's. #[tokio::test] async fn decrypt_refuses_a_value_that_opens_to_another_type() { let cipher = cipher().await; let keyset = cipher.default_keyset(); - let untyped = plan(obj(vec![("age", spec(s("users/age"), &["c"]))])).expect("plan"); - let sealed = encrypt(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped) - .await - .expect("seal"); - let as_uint64 = - plan(obj(vec![("age", typed("users/age", &["c"], "uint64"))])).expect("plan"); - let result = decrypt(Scope::Client(&cipher), sealed, &as_uint64).await; - assert!(matches!(result, Err(Error::Record)), "{:?}", result.err()); + let untyped = plan(obj(vec![("age", spec(label("age"), &["c"]))])).expect("plan"); + let sealed = seal(&keyset, obj(vec![("age", FfiValue::UInt64(34))]), &untyped).await; + let as_int64 = + plan(obj(vec![("age", typed(label("age"), &["c"], "int64"))])).expect("plan"); + let result = decrypt(Scope::Client(&cipher), sealed, &as_int64) + .expect("the shape fits") + .await; + assert_eq!( + plan_error(result.err().expect("refused")), + PlanError::FieldType { + field: "age".into(), + expected: "int64", + } + ); - let sealed = encrypt(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped) - .await - .expect("seal"); - let as_uint32 = - plan(obj(vec![("age", typed("users/age", &["c"], "uint32"))])).expect("plan"); - let opened = decrypt(Scope::Client(&cipher), sealed, &as_uint32) - .await - .expect("the declared type opens"); - assert_eq!(u32_of(&object(opened)[0].1), 34); + let sealed = seal(&keyset, obj(vec![("age", FfiValue::UInt64(34))]), &untyped).await; + let as_uint64 = + plan(obj(vec![("age", typed(label("age"), &["c"], "uint64"))])).expect("plan"); + let opened = open(&cipher, sealed, &as_uint64).await; + assert_eq!(u64_of(&object(opened)[0].1), 34, "the declared type opens"); } /// In a batch, each opened value is checked against its own field, @@ -2538,24 +3275,20 @@ mod tests { let cipher = cipher().await; let keyset = cipher.default_keyset(); let plan = plan(obj(vec![ - ("age", typed("users/age", &["c"], "uint32")), - ("name", typed("users/name", &["c"], "string")), + ("age", typed(label("age"), &["c"], "uint64")), + ("name", typed(label("name"), &["c"], "string")), ])) .expect("plan"); let rows = FfiValue::Array(vec![ - obj(vec![("age", FfiValue::UInt32(1)), ("name", s("a"))]), - obj(vec![("age", FfiValue::UInt32(2)), ("name", s("b"))]), + obj(vec![("age", FfiValue::UInt64(1)), ("name", s("a"))]), + obj(vec![("age", FfiValue::UInt64(2)), ("name", s("b"))]), ]); - let sealed = encrypt(&keyset, rows, &plan).await.expect("seal"); - let opened = array( - decrypt(Scope::Client(&cipher), sealed, &plan) - .await - .expect("open"), - ); + let sealed = seal(&keyset, rows, &plan).await; + let opened = array(open(&cipher, sealed, &plan).await); assert_eq!(opened.len(), 2); let second = object(opened.into_iter().nth(1).expect("row")); assert_eq!(keys(&second), ["age", "name"]); - assert_eq!(u32_of(&second[0].1), 2); + assert_eq!(u64_of(&second[0].1), 2); assert_eq!(text_of(&second[1].1), "b"); } @@ -2567,9 +3300,7 @@ mod tests { let keyset = cipher.default_keyset(); let plan = age_plan("uint64"); let field = &plan.fields()[0]; - let sealed = encrypt(&keyset, obj(vec![("age", FfiValue::UInt64(34))]), &plan) - .await - .expect("seal"); + let sealed = seal(&keyset, obj(vec![("age", FfiValue::UInt64(34))]), &plan).await; let mut row = map(sealed); let mut age = map(node(&mut row, "age")); let stored = term_bytes(&node(&mut age, "eq")); @@ -2582,7 +3313,7 @@ mod tests { &keyset, scalar, &IndexSpec::Equality, - field.view().expect("view"), + field.context().clone(), ) .await .expect("query term"); @@ -2597,22 +3328,17 @@ mod tests { let cipher = cipher().await; let keyset = cipher.default_keyset(); let plan = plan(obj(vec![ - ("age", typed("users/age", &["eq"], "uint64")), - ("name", typed("users/name", &["c"], "string")), + ("age", typed(label("age"), &["eq"], "uint64")), + ("name", typed(label("name"), &["c"], "string")), ])) .expect("plan"); - let sealed = encrypt( + let sealed = seal( &keyset, obj(vec![("age", FfiValue::UInt64(34)), ("name", s("bob"))]), &plan, ) - .await - .expect("seal"); - let opened = object( - decrypt(Scope::Client(&cipher), sealed, &plan) - .await - .expect("name is checked as a string, not as a uint64"), - ); + .await; + let opened = object(open(&cipher, sealed, &plan).await); assert_eq!(keys(&opened), ["name"]); assert_eq!(text_of(&opened[0].1), "bob"); } @@ -2634,13 +3360,9 @@ mod tests { ), ("array", FfiValue::Array(vec![]), 0), ] { - let plan = plan(obj(vec![("doc", typed("users/doc", &["c"], ty))])).expect("plan"); - let sealed = encrypt(&keyset, obj(vec![("doc", value)]), &plan) - .await - .expect("seal"); - let opened = decrypt(Scope::Client(&cipher), sealed, &plan) - .await - .expect("a composite opens as its declared kind"); + let plan = plan(obj(vec![("doc", typed(label("doc"), &["c"], ty))])).expect("plan"); + let sealed = seal(&keyset, obj(vec![("doc", value)]), &plan).await; + let opened = open(&cipher, sealed, &plan).await; let (_, doc) = object(opened).into_iter().next().expect("doc"); match (ty, doc) { ("object", FfiValue::Object(entries)) => assert_eq!(entries.len(), len), @@ -2651,30 +3373,208 @@ mod tests { } /// `check_record` does not open anything, so it cannot see a - /// typed field's type: the tag is inside the AEAD envelope. A record - /// sealed as another type passes it, and only `decrypt` refuses it. + /// typed field's type: the tag is inside the AEAD envelope. A + /// record sealed as another type passes it, and only awaiting + /// `decrypt` refuses it. #[tokio::test] async fn check_record_accepts_a_record_sealed_as_another_type() { let cipher = cipher().await; let keyset = cipher.default_keyset(); - let untyped = plan(obj(vec![("age", spec(s("users/age"), &["c"]))])).expect("plan"); + let untyped = plan(obj(vec![("age", spec(label("age"), &["c"]))])).expect("plan"); let as_uint64 = - plan(obj(vec![("age", typed("users/age", &["c"], "uint64"))])).expect("plan"); + plan(obj(vec![("age", typed(label("age"), &["c"], "uint64"))])).expect("plan"); - let sealed = encrypt(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped) - .await - .expect("seal"); + let sealed = seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped).await; check_record(sealed, &as_uint64).expect("the type is not visible without opening"); - let sealed = encrypt(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped) - .await - .expect("seal"); - let result = decrypt(Scope::Client(&cipher), sealed, &as_uint64).await; + let sealed = seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped).await; + let result = decrypt(Scope::Client(&cipher), sealed, &as_uint64) + .expect("the shape fits") + .await; assert!( - matches!(result, Err(Error::Record)), + matches!(result, Err(crate::Error::Plan(PlanError::FieldType { .. }))), "decrypt is where the type is checked: {:?}", result.err() ); } + + /// A passthrough field with a declared type carries only values of + /// that type, in and out. + #[tokio::test] + async fn a_typed_passthrough_field_is_checked_both_ways() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let plan = plan(obj(vec![ + ("age", typed(label("age"), &["c"], "uint32")), + ("id", typed(label("id"), &["passthrough"], "uint64")), + ])) + .expect("plan"); + let refused = encrypt( + &keyset, + obj(vec![ + ("age", FfiValue::UInt32(1)), + ("id", FfiValue::UInt32(7)), + ]), + &plan, + ) + .err(); + assert!(matches!(refused, Some(Error::Source)), "{refused:?}"); + let sealed = seal( + &keyset, + obj(vec![ + ("age", FfiValue::UInt32(1)), + ("id", FfiValue::UInt64(7)), + ]), + &plan, + ) + .await; + let mut fields = map(sealed); + let age = node(&mut fields, "age"); + fields.push(("age".to_string(), age)); + let _ = node(&mut fields, "id"); + fields.push(( + "id".to_string(), + CipherText::Map(vec![( + "passthrough".to_string(), + forged(FfiValue::UInt32(7)), + )]), + )); + let refused = decrypt(Scope::Client(&cipher), CipherText::Map(fields), &plan).err(); + assert!(matches!(refused, Some(Error::Record)), "{refused:?}"); + } + } + + /// The leaf encoding decision, pinned (ADR-0007): a field typed as a + /// kind with a Rust leaf type seals as that type seals, so a data plan + /// and a derive interchange; a field with no type seals the tagged + /// `FfiValue` encoding, which only a dynamic reader opens. + mod given_the_leaf_encoding { + use super::*; + + #[derive(EncryptFrom, DecryptInto)] + #[stash(plaintext = u32, crate = "crate")] + struct Age { + c: StackCipherText, + hm: EqualityTerm, + } + + /// A derived `plaintext = u32` record and a `uint32` data-plan field + /// under the same label: the same term, and each opens the other's + /// ciphertext. + #[tokio::test] + async fn a_typed_field_and_the_derive_interchange() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let label_ = Label::parse("users/age").expect("label"); + let derived: Age = keyset + .encrypt_as(&34u32, CallerContext::from(NonEmpty::from(label_.clone()))) + .await + .expect("derive"); + + let plan = plan(obj(vec![( + "age", + typed(label("age"), &["c", "eq"], "uint32"), + )])) + .expect("plan"); + let mut fields = + map(seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &plan).await); + let mut age = map(node(&mut fields, "age")); + assert_eq!( + term_bytes(&node(&mut age, "eq")), + derived.hm.to_bytes(), + "the same equality term" + ); + + // The derive opens the lowering's leaf as a bare u32. + let opened: u32 = keyset + .decrypt_as( + node(&mut age, "c"), + AeadContext::from(NonEmpty::from(label_.clone())), + ) + .await + .expect("a bare u32 leaf"); + assert_eq!(opened, 34); + + // The lowering opens the derive's leaf as a uint32 field. + let stored = CipherText::Map(vec![( + "age".to_string(), + CipherText::Map(vec![("c".to_string(), derived.c)]), + )]); + let opened = object(open(&cipher, stored, &plan).await); + assert_eq!(u32_of(&opened[0].1), 34); + } + + /// A field with no type seals the self-describing tagged encoding: + /// it reads back as what it was, and a bare `u32` reader does not + /// open it. The two encodings are different leaves, by declaration. + #[tokio::test] + async fn an_untyped_field_seals_the_tagged_encoding() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let label_ = Label::parse("users/age").expect("label"); + let untyped = plan(obj(vec![("age", spec(label("age"), &["c"]))])).expect("plan"); + let mut fields = + map(seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped).await); + let mut age = map(node(&mut fields, "age")); + let as_value: FfiValue = cipher + .decrypt(node(&mut age, "c"), label_.clone()) + .await + .expect("the tagged leaf opens as a value"); + assert_eq!(u32_of(&as_value), 34); + let mut fields = + map(seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped).await); + let mut age = map(node(&mut fields, "age")); + let as_u32: Result = cipher.decrypt(node(&mut age, "c"), label_).await; + assert!( + as_u32.is_err(), + "five tagged bytes are not a bare u32: {as_u32:?}" + ); + + // And the typed leaf is not the tagged one. + let typed_plan = + plan(obj(vec![("age", typed(label("age"), &["c"], "uint32"))])).expect("plan"); + let mut fields = map(seal( + &keyset, + obj(vec![("age", FfiValue::UInt32(34))]), + &typed_plan, + ) + .await); + let mut age = map(node(&mut fields, "age")); + let as_value: Result = cipher + .decrypt( + node(&mut age, "c"), + Label::parse("users/age").expect("label"), + ) + .await; + assert!( + as_value.is_err(), + "four bare bytes carry no tag for a value reader" + ); + } + + /// A `string` field is a `String` leaf, as the derive's. + #[tokio::test] + async fn a_string_field_is_a_string_leaf() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let plan = plan(obj(vec![( + "email", + typed(label("email"), &["c", "eq"], "string"), + )])) + .expect("plan"); + let mut fields = map(seal(&keyset, obj(vec![("email", s("a@x"))]), &plan).await); + let mut email = map(node(&mut fields, "email")); + let label_ = Label::parse("users/email").expect("label"); + let opened: String = cipher + .decrypt(node(&mut email, "c"), label_.clone()) + .await + .expect("a bare string leaf"); + assert_eq!(opened, "a@x"); + let typed = keyset + .equality_term("a@x".to_string(), NonEmpty::from(label_)) + .await + .expect("typed"); + assert_eq!(term_bytes(&node(&mut email, "eq")), typed.to_bytes()); + } } } diff --git a/packages/stack-encrypt/src/dynamic/term.rs b/packages/stack-encrypt/src/dynamic/term.rs index 09af986ca..1eecc49b1 100644 --- a/packages/stack-encrypt/src/dynamic/term.rs +++ b/packages/stack-encrypt/src/dynamic/term.rs @@ -19,9 +19,12 @@ use vitaminc_aead_value::FfiValue; use vitaminc_protected::{Controlled, OpaqueDebug, Protected}; use zeroize::Zeroizing; -use super::{utf8, Error}; +use super::{utf8, Error, Value}; use crate::sem::{CllwOpeEncrypt, CllwOreEncrypt, DefaultMatch, MatchOptions, Tokenizer}; -use crate::target::IndexSpec; +use crate::target::{ + chosen, equality, ope as ope_op, ore as ore_op, CallerContext, ConsumeSource, Encryption, + Index, IndexSpec, Pending, +}; use crate::{IntoPrfContext, KeysetCipher, NonEmpty}; /// The runtime half of [`IndexSpec`]: the domain table, and the index's wire @@ -271,6 +274,11 @@ impl Scalar { /// # }).unwrap(); /// ``` /// +/// The derivation is `scalar_term`'s, the one dispatch from a runtime +/// scalar to the typed term operation; the record path runs the same +/// dispatch as an [`Index`] of a [`Value`] field. It is local to the keyset +/// cipher, so the await settles a pending that has nothing to request. +/// /// # Errors /// /// [`Error::Term`] if the scheme defines no such term for the scalar @@ -284,39 +292,93 @@ pub async fn term<'c, K, D>( context: NonEmpty, ) -> Result, Error> where - K: DataKeySource + Sync, + K: DataKeySource + Sync + 'static, D: IntoPrfContext<'c>, { - match kind { - IndexSpec::Equality => equality(cipher, scalar, context).await, - IndexSpec::Match(options) => match_term(cipher, scalar, options, context).await, - IndexSpec::Ore => ore_of(cipher, scalar, context).await, - IndexSpec::Ope => ope_of(cipher, scalar, context).await, + scalar_term(cipher, scalar, kind, context)? + .await + .map(TermBytes::into_bytes) + .map_err(Error::Cipher) +} + +/// An index term in its frozen byte encoding, as a binding stores and +/// compares it: the raw 32 PRF bytes of an equality term, a match term's +/// positions, the raw CLLW bytes of an ORE or OPE term (see +/// [`sem`](crate::sem)'s byte encodings). +/// +/// The term type of every [`Index`] an [`IndexSpec`] implements: a term +/// derived through the dynamic path has no Rust term type to be, since the +/// index was named as data, so it is its bytes. Those bytes are exactly the +/// typed term's (`EqualityTerm::into_bytes`, `OreTerm::to_bytes`, …), +/// which is what makes a Rust-written term and a binding's probe compare. +#[derive(Clone, PartialEq, Eq, Debug)] +pub struct TermBytes(Vec); + +impl TermBytes { + /// The bytes. + pub fn as_bytes(&self) -> &[u8] { + &self.0 + } + + /// The bytes, owned. + pub fn into_bytes(self) -> Vec { + self.0 } } +impl AsRef<[u8]> for TermBytes { + fn as_ref(&self) -> &[u8] { + &self.0 + } +} + +/// The one dispatch from a runtime scalar to a typed term operation: which +/// arm a scalar takes decides the term's input encoding, and each arm is +/// the operation a Rust caller would have named for that type, so the +/// bytes are the typed path's. Nothing is requested: a term is derived by +/// the keyset cipher's own PRF, so the pending is ready when it is built. +/// +/// # Errors +/// +/// [`Error::Term`] if the scheme defines no such term for the scalar. +pub(super) fn scalar_term<'a, 'c, K: 'static, D>( + cipher: &'a KeysetCipher<'_, K>, + scalar: Scalar, + kind: &IndexSpec, + context: NonEmpty, +) -> Result, Error> +where + D: IntoPrfContext<'c>, +{ + Ok(match kind { + IndexSpec::Equality => equality_of(cipher, scalar, context)?, + IndexSpec::Match(options) => match_of(cipher, scalar, options, context)?, + IndexSpec::Ore => ore_of(cipher, scalar, context), + IndexSpec::Ope => ope_of(cipher, scalar, context), + }) +} + +fn equality_bytes(term: crate::sem::EqualityTerm) -> TermBytes { + TermBytes(term.into_bytes().to_vec()) +} + /// [`IndexSpec::Equality`] per scalar: one PRF block over the value, for /// every integer width, text and bytes. -async fn equality<'c, K, D>( - cipher: &KeysetCipher<'_, K>, +fn equality_of<'a, 'c, K: 'static, D>( + cipher: &'a KeysetCipher<'_, K>, scalar: Scalar, context: NonEmpty, -) -> Result, Error> +) -> Result, Error> where - K: DataKeySource + Sync, D: IntoPrfContext<'c>, { let term = match scalar { - Scalar::I32(v) => cipher.equality_term(v, context).await, - Scalar::I64(v) => cipher.equality_term(v, context).await, - Scalar::U32(v) => cipher.equality_term(v, context).await, - Scalar::U64(v) => cipher.equality_term(v, context).await, - Scalar::Text(t) => cipher.equality_term(String::clone(&t), context).await, - Scalar::Bytes(b) => { - cipher - .equality_term(Protected::new(Vec::clone(&b)), context) - .await - } + Scalar::I32(v) => cipher.equality_term(v, context), + Scalar::I64(v) => cipher.equality_term(v, context), + Scalar::U32(v) => cipher.equality_term(v, context), + Scalar::U64(v) => cipher.equality_term(v, context), + Scalar::Text(t) => cipher.equality_term(String::clone(&t), context), + Scalar::Bytes(b) => cipher.equality_term(Protected::new(Vec::clone(&b)), context), // No PRF encoding is defined for floats (equality on IEEE-754 // values is a modelling error) or booleans. Scalar::Bool(_) | Scalar::F32(_) | Scalar::F64(_) => { @@ -324,29 +386,27 @@ where kind: IndexSpec::Equality, }) } - }?; - Ok(term.into_bytes().to_vec()) + }; + Ok(term.map(equality_bytes)) } /// [`IndexSpec::Match`] per scalar: text only, under the index's options. /// Under the default options the bytes are the typed /// `match_terms::`'s; under others, those of a /// [`MatchConfig`](crate::sem::MatchConfig) returning the same options. -async fn match_term<'c, K, D>( - cipher: &KeysetCipher<'_, K>, +fn match_of<'a, 'c, K: 'static, D>( + cipher: &'a KeysetCipher<'_, K>, scalar: Scalar, options: &MatchOptions, context: NonEmpty, -) -> Result, Error> +) -> Result, Error> where - K: DataKeySource + Sync, D: IntoPrfContext<'c>, { match scalar { Scalar::Text(t) => Ok(cipher .match_terms_under::(&t, context, options.clone()) - .await - .map(|t| t.to_bytes())?), + .map(|terms| TermBytes(terms.to_bytes()))), _ => Err(Error::Term { kind: IndexSpec::Match(options.clone()), }), @@ -361,87 +421,178 @@ where /// cloned-out `String`/`Vec` would be freed with the plaintext /// still in it — in a guest's linear memory, where the host can read /// it. Keeping the wrapper costs nothing and saves the copy as well. -async fn ore_of<'c, K, D>( - cipher: &KeysetCipher<'_, K>, +fn ore_of<'a, 'c, K: 'static, D>( + cipher: &'a KeysetCipher<'_, K>, scalar: Scalar, context: NonEmpty, -) -> Result, Error> +) -> Pending<'a, TermBytes, K> where - K: DataKeySource + Sync, D: IntoPrfContext<'c>, { match scalar { - Scalar::Bool(v) => ore(cipher, v, context).await, - Scalar::I32(v) => ore(cipher, v, context).await, - Scalar::I64(v) => ore(cipher, v, context).await, - Scalar::U32(v) => ore(cipher, v, context).await, - Scalar::U64(v) => ore(cipher, v, context).await, - Scalar::F32(v) => ore(cipher, v, context).await, - Scalar::F64(v) => ore(cipher, v, context).await, - Scalar::Text(t) => ore(cipher, t, context).await, - Scalar::Bytes(b) => ore(cipher, b, context).await, + Scalar::Bool(v) => ore(cipher, v, context), + Scalar::I32(v) => ore(cipher, v, context), + Scalar::I64(v) => ore(cipher, v, context), + Scalar::U32(v) => ore(cipher, v, context), + Scalar::U64(v) => ore(cipher, v, context), + Scalar::F32(v) => ore(cipher, v, context), + Scalar::F64(v) => ore(cipher, v, context), + Scalar::Text(t) => ore(cipher, t, context), + Scalar::Bytes(b) => ore(cipher, b, context), } } /// [`IndexSpec::Ope`] per scalar; see [`ore_of`] for why the text and bytes /// arms pass the wrapper. -async fn ope_of<'c, K, D>( - cipher: &KeysetCipher<'_, K>, +fn ope_of<'a, 'c, K: 'static, D>( + cipher: &'a KeysetCipher<'_, K>, scalar: Scalar, context: NonEmpty, -) -> Result, Error> +) -> Pending<'a, TermBytes, K> where - K: DataKeySource + Sync, D: IntoPrfContext<'c>, { match scalar { - Scalar::Bool(v) => ope(cipher, v, context).await, - Scalar::I32(v) => ope(cipher, v, context).await, - Scalar::I64(v) => ope(cipher, v, context).await, - Scalar::U32(v) => ope(cipher, v, context).await, - Scalar::U64(v) => ope(cipher, v, context).await, - Scalar::F32(v) => ope(cipher, v, context).await, - Scalar::F64(v) => ope(cipher, v, context).await, - Scalar::Text(t) => ope(cipher, t, context).await, - Scalar::Bytes(b) => ope(cipher, b, context).await, + Scalar::Bool(v) => ope(cipher, v, context), + Scalar::I32(v) => ope(cipher, v, context), + Scalar::I64(v) => ope(cipher, v, context), + Scalar::U32(v) => ope(cipher, v, context), + Scalar::U64(v) => ope(cipher, v, context), + Scalar::F32(v) => ope(cipher, v, context), + Scalar::F64(v) => ope(cipher, v, context), + Scalar::Text(t) => ope(cipher, t, context), + Scalar::Bytes(b) => ope(cipher, b, context), } } /// The `AsRef<[u8]>` on the output is what turns the typed CLLW ciphertext /// into the frozen raw-bytes encoding. -async fn ore<'c, K, T, D>( - cipher: &KeysetCipher<'_, K>, +fn ore<'a, 'c, K: 'static, T, D>( + cipher: &'a KeysetCipher<'_, K>, value: T, context: NonEmpty, -) -> Result, Error> +) -> Pending<'a, TermBytes, K> where - K: DataKeySource + Sync, T: CllwOreEncrypt + Send + 'static, T::Output: AsRef<[u8]> + Send + 'static, D: IntoPrfContext<'c>, { - Ok(cipher + cipher .ore_term(value, context) - .await - .map(|t| t.as_ref().to_vec())?) + .map(|term| TermBytes(term.as_ref().to_vec())) } /// See [`ore`]. -async fn ope<'c, K, T, D>( - cipher: &KeysetCipher<'_, K>, +fn ope<'a, 'c, K: 'static, T, D>( + cipher: &'a KeysetCipher<'_, K>, value: T, context: NonEmpty, -) -> Result, Error> +) -> Pending<'a, TermBytes, K> where - K: DataKeySource + Sync, T: CllwOpeEncrypt + Send + 'static, T::Output: AsRef<[u8]> + Send + 'static, D: IntoPrfContext<'c>, { - Ok(cipher + cipher .ope_term(value, context) - .await - .map(|t| t.as_ref().to_vec())?) + .map(|term| TermBytes(term.as_ref().to_vec())) +} + +/// A dynamic error, where an operation's pending can only carry the crate's. +fn lifted(error: Error) -> crate::Error { + crate::Error::Other(Box::new(error)) +} + +/// An [`IndexSpec`] is an [`Index`] of a [`Value`]: the index named as data, +/// over a plaintext whose type is known only when it arrives. Its +/// `operation` is `scalar_term`'s dispatch — the one step of the dynamic +/// path that stays dynamic — wrapped as a description, so a field lowered +/// from data runs through the same `indexed()` and `zip` every other field +/// does, and its term is a [`TermBytes`]. +/// +/// A value the scheme defines no such term for (a container, a float under +/// equality) fails the description when it runs; a plan lowered from data +/// refuses it before that, at its boundary ([`IndexSpec::supports`]). +impl Index for IndexSpec { + type Term = TermBytes; + fn spec(&self) -> IndexSpec { + self.clone() + } + fn operation<'s, K: 'static, M: ConsumeSource<'s, Value>>( + &self, + ) -> Encryption<'s, Value, TermBytes, K, CallerContext, M> { + let spec = self.clone(); + chosen(move |source: M::Source, cipher, cx: CallerContext| { + let scalar = match Scalar::of(M::view(&source).get(), &spec) { + Ok(scalar) => scalar, + Err(error) => return Pending::failed(cipher, lifted(error)), + }; + let context = match cx.validated() { + Ok(context) => context, + Err(error) => return Pending::failed(cipher, error), + }; + match scalar_term(cipher, scalar, &spec, context) { + Ok(pending) => pending, + Err(error) => Pending::failed(cipher, lifted(error)), + } + }) + } +} + +/// An [`IndexSpec`] is an [`Index`] of a `u32`: the typed index of the same +/// name — [`equality`], [`ore`](crate::target::ore), [`ope`](crate::target::ope) — +/// with its term as [`TermBytes`]. This is how a plan lowered from data runs +/// a field it knows to be a `u32` through exactly the operations the typed +/// chain's `encrypt_index::` runs. Match is not defined over an +/// integer, and fails the description when it runs; a plan refuses it when +/// it is built ([`admits`](super::admits)). +impl Index for IndexSpec { + type Term = TermBytes; + fn spec(&self) -> IndexSpec { + self.clone() + } + fn operation<'s, K: 'static, M: ConsumeSource<'s, u32>>( + &self, + ) -> Encryption<'s, u32, TermBytes, K, CallerContext, M> { + match self { + IndexSpec::Equality => equality::().map(equality_bytes), + IndexSpec::Ore => ore_op::().map(|term| TermBytes(term.to_bytes())), + IndexSpec::Ope => ope_op::().map(|term| TermBytes(term.to_bytes())), + IndexSpec::Match(_) => Encryption::failed(lifted(Error::Term { kind: self.clone() })), + } + } +} + +/// An [`IndexSpec`] is an [`Index`] of a `String`: the typed index of the +/// same name, with its term as [`TermBytes`]; see the `u32` impl. A match +/// index derives under the options the spec carries, which under the +/// defaults are `Match::default()`'s bytes. +impl Index for IndexSpec { + type Term = TermBytes; + fn spec(&self) -> IndexSpec { + self.clone() + } + fn operation<'s, K: 'static, M: ConsumeSource<'s, String>>( + &self, + ) -> Encryption<'s, String, TermBytes, K, CallerContext, M> { + match self { + IndexSpec::Equality => equality::().map(equality_bytes), + IndexSpec::Ore => ore_op::().map(|term| TermBytes(term.to_bytes())), + IndexSpec::Ope => ope_op::().map(|term| TermBytes(term.to_bytes())), + IndexSpec::Match(options) => { + let options = options.clone(); + chosen(move |source: M::Source, cipher, cx: CallerContext| { + let context = match cx.validated() { + Ok(context) => context, + Err(error) => return Pending::failed(cipher, error), + }; + cipher + .match_terms_under::(M::view(&source), context, options) + .map(|terms| TermBytes(terms.to_bytes())) + }) + } + } + } } #[cfg(test)] diff --git a/packages/stack-encrypt/src/dynamic/value.rs b/packages/stack-encrypt/src/dynamic/value.rs new file mode 100644 index 000000000..3e7079564 --- /dev/null +++ b/packages/stack-encrypt/src/dynamic/value.rs @@ -0,0 +1,199 @@ +//! An [`FfiValue`] as a plan field. See [`Value`]. + +use std::fmt; + +use vitaminc_aead::{Cipher, Decipher, Decrypt, Encrypt, IntoAad}; +use vitaminc_aead_value::FfiValue; +use vitaminc_protected::{Controlled, Protected}; + +/// A runtime value as the plaintext of a plan field: what a field lowered +/// from data holds when its declared type names no Rust leaf type, or when +/// it declares none. +/// +/// The engine hands a borrowed field to every operation that consumes it +/// and clones it on the way in, as it does a `String` field of a derived +/// record, so a field's plaintext type is `Clone`. [`FfiValue`] is not: +/// vitaminc keeps its leaves in [`Protected`] and gives no copy out +/// implicitly. This wrapper is the copy made explicit, once, here: its +/// `Clone` rebuilds the tree leaf by leaf into fresh `Protected` payloads, +/// which wipe on drop as the originals do, so a clone is under the same +/// custody as the value it was made from. A record of a few fields clones +/// each once per operation that consumes it, as the derive's does. +/// +/// It seals and opens as the [`FfiValue`] it wraps, in vitaminc's +/// self-describing tagged leaf encoding; [`dynamic::record`](super::record) +/// says when a field is this type and when it is a bare Rust leaf instead. +/// Its `Debug` names the type and nothing else: the value is plaintext. +pub struct Value(FfiValue); + +impl Value { + /// Wrap a value. + pub fn new(value: FfiValue) -> Self { + Self(value) + } + + /// The value. + pub fn get(&self) -> &FfiValue { + &self.0 + } + + /// Unwrap the value. + pub fn into_inner(self) -> FfiValue { + self.0 + } +} + +impl From for Value { + fn from(value: FfiValue) -> Self { + Self(value) + } +} + +impl From for FfiValue { + fn from(value: Value) -> Self { + value.0 + } +} + +impl fmt::Debug for Value { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str("Value") + } +} + +impl Clone for Value { + fn clone(&self) -> Self { + Self(duplicate(&self.0)) + } +} + +/// A deep copy of a value, each byte-bearing leaf into a fresh +/// [`Protected`]. Exhaustive, so a variant added to [`FfiValue`] has to say +/// how it is copied. +fn duplicate(value: &FfiValue) -> FfiValue { + match value { + FfiValue::Null => FfiValue::Null, + FfiValue::Undefined => FfiValue::Undefined, + FfiValue::Bool(v) => FfiValue::Bool(*v), + FfiValue::Int32(v) => FfiValue::Int32(*v), + FfiValue::Int64(v) => FfiValue::Int64(*v), + FfiValue::UInt32(v) => FfiValue::UInt32(*v), + FfiValue::UInt64(v) => FfiValue::UInt64(*v), + FfiValue::Float32(v) => FfiValue::Float32(*v), + FfiValue::Float64(v) => FfiValue::Float64(*v), + // Valid UTF-8 by `Utf8String`'s construction invariant. Were it not, + // the copy is a bytes leaf: a different type the plan then refuses, + // never a string that reads differently from its original. + FfiValue::String(s) => match std::str::from_utf8(s.risky_ref()) { + Ok(text) => FfiValue::String(text.into()), + Err(_) => FfiValue::Bytes(Protected::new(s.risky_ref().to_vec())), + }, + FfiValue::Bytes(b) => FfiValue::Bytes(Protected::new(b.risky_ref().to_vec())), + FfiValue::Array(items) => FfiValue::Array(items.iter().map(duplicate).collect()), + FfiValue::Object(entries) => FfiValue::Object( + entries + .iter() + .map(|(key, value)| (key.clone(), duplicate(value))) + .collect(), + ), + FfiValue::Passthrough(inner) => FfiValue::Passthrough(Box::new(duplicate(inner))), + } +} + +impl Encrypt for Value { + fn encrypt_with_aad<'a, C, A>(self, cipher: C, aad: A) -> Result + where + C: Cipher, + A: IntoAad<'a>, + { + self.0.encrypt_with_aad(cipher, aad) + } +} + +impl<'c> Decrypt<'c> for Value { + fn decrypt_with_aad<'a, D, A>(decipher: D, aad: A) -> D::Ok + where + D: Decipher<'c>, + A: IntoAad<'a>, + { + D::map_ok(FfiValue::decrypt_with_aad(decipher, aad), Value) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn s(value: &str) -> FfiValue { + FfiValue::String(value.into()) + } + + /// Structural equality, which `FfiValue` offers only inside vitaminc's + /// own tests. + fn same(a: &FfiValue, b: &FfiValue) -> bool { + match (a, b) { + (FfiValue::Null, FfiValue::Null) | (FfiValue::Undefined, FfiValue::Undefined) => true, + (FfiValue::Bool(x), FfiValue::Bool(y)) => x == y, + (FfiValue::Int32(x), FfiValue::Int32(y)) => x == y, + (FfiValue::Int64(x), FfiValue::Int64(y)) => x == y, + (FfiValue::UInt32(x), FfiValue::UInt32(y)) => x == y, + (FfiValue::UInt64(x), FfiValue::UInt64(y)) => x == y, + (FfiValue::Float32(x), FfiValue::Float32(y)) => x.to_bits() == y.to_bits(), + (FfiValue::Float64(x), FfiValue::Float64(y)) => x.to_bits() == y.to_bits(), + (FfiValue::String(x), FfiValue::String(y)) => x.risky_ref() == y.risky_ref(), + (FfiValue::Bytes(x), FfiValue::Bytes(y)) => x.risky_ref() == y.risky_ref(), + (FfiValue::Array(x), FfiValue::Array(y)) => { + x.len() == y.len() && x.iter().zip(y).all(|(a, b)| same(a, b)) + } + (FfiValue::Object(x), FfiValue::Object(y)) => { + x.len() == y.len() + && x.iter() + .zip(y) + .all(|((ka, va), (kb, vb))| ka == kb && same(va, vb)) + } + (FfiValue::Passthrough(x), FfiValue::Passthrough(y)) => same(x, y), + _ => false, + } + } + + /// A clone is the same value, leaf for leaf, at every depth and for + /// every variant, and shares no payload with its original. + #[test] + fn a_clone_is_the_same_value_in_fresh_payloads() { + let original = Value::new(FfiValue::Object(vec![ + ("n".to_string(), FfiValue::Null), + ("u".to_string(), FfiValue::Undefined), + ("b".to_string(), FfiValue::Bool(true)), + ("i32".to_string(), FfiValue::Int32(-3)), + ("i64".to_string(), FfiValue::Int64(-4)), + ("u32".to_string(), FfiValue::UInt32(34)), + ("u64".to_string(), FfiValue::UInt64(35)), + ("f32".to_string(), FfiValue::Float32(1.5)), + ("f64".to_string(), FfiValue::Float64(2.5)), + ("s".to_string(), s("alice")), + ( + "bytes".to_string(), + FfiValue::Bytes(Protected::new(vec![1, 2, 3])), + ), + ( + "list".to_string(), + FfiValue::Array(vec![s("a"), FfiValue::Passthrough(Box::new(s("p")))]), + ), + ])); + let copy = original.clone(); + assert!(same(copy.get(), original.get()), "the same value"); + let (FfiValue::Object(a), FfiValue::Object(b)) = (original.get(), copy.get()) else { + panic!("objects"); + }; + let (FfiValue::String(x), FfiValue::String(y)) = (&a[9].1, &b[9].1) else { + panic!("strings"); + }; + assert_ne!( + x.risky_ref().as_ptr(), + y.risky_ref().as_ptr(), + "the text was copied, not shared" + ); + assert_eq!(format!("{copy:?}"), "Value", "debug shows no plaintext"); + assert!(matches!(copy.into_inner(), FfiValue::Object(_))); + } +} diff --git a/packages/stack-encrypt/src/plan/error.rs b/packages/stack-encrypt/src/plan/error.rs index 6f6c04154..ec32afe65 100644 --- a/packages/stack-encrypt/src/plan/error.rs +++ b/packages/stack-encrypt/src/plan/error.rs @@ -69,6 +69,12 @@ pub enum PlanError { /// The index named twice, as its key (`"eq"`, `"match"`, ...). index: &'static str, }, + /// A field was declared indexed with an index set that holds no index. + /// A tuple of indexes cannot be empty, so this is only reachable through + /// an index set sized at run time (a `Vec`), as a plan lowered from data + /// builds. + #[error("an indexed field declares no index")] + EmptyIndexes, /// The value has a field the plan does not name. #[error("the value has a field {field:?} the plan does not name")] NotInPlan { diff --git a/packages/stack-encrypt/src/plan/mod.rs b/packages/stack-encrypt/src/plan/mod.rs index f09b76bfe..9baa2ffbf 100644 --- a/packages/stack-encrypt/src/plan/mod.rs +++ b/packages/stack-encrypt/src/plan/mod.rs @@ -259,6 +259,16 @@ //! # tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap().block_on(example()).unwrap(); //! ``` //! +//! # A binding lowers data into the same builder +//! +//! A language binding has no types to name, so it declares a record as +//! data. With the `dynamic` feature, [`dynamic::record`](crate::dynamic::record) +//! reads that declaration and lowers it into this builder — one context, +//! `encrypt` / `encrypt_index` / `index` / `passthrough` per field — and +//! runs the result through [`KeysetCipher::run`](crate::KeysetCipher::run) +//! and the plan's opener. It is the third author of this grammar, beside a +//! Rust chain and the derive, and not a second executor. +//! //! # How a chain lowers //! //! The builder adds no cryptographic operation and no executor; every call diff --git a/packages/stack-encrypt/src/target/context.rs b/packages/stack-encrypt/src/target/context.rs index 48ad1cd3b..b21f6dfee 100644 --- a/packages/stack-encrypt/src/target/context.rs +++ b/packages/stack-encrypt/src/target/context.rs @@ -46,9 +46,18 @@ impl<'a> IntoContext<'a> for CallerContext { } } impl CallerContext { - pub(super) fn validated(self) -> Result, Error> { + pub(crate) fn validated(self) -> Result, Error> { nonempty(self) } + /// A caller context from a context part as a binding spelled it, with + /// no emptiness proof: what a plan lowered from data extends a field's + /// context by. An empty part extends a nonempty context to a nonempty + /// one, so the proof it skips is the whole tree's, which + /// [`DeclaredContext::under`] re-establishes when it runs. + #[cfg(feature = "dynamic")] + pub(crate) fn from_piece(piece: ContextPiece<'static>) -> Self { + Self(piece) + } /// The own context `own`, extended by this caller context: the field's /// own context is the prefix, this context the extension, exactly as a /// `struct = T` derive composes them — `(("users", "age"), id)`. The own @@ -139,8 +148,14 @@ impl Extends for AeadContext { /// value extends each of them, the way a caller's context extends a field's /// own. This is what a `struct = T` derive without a `context_field` /// declares. +/// +/// An extension is one part or several, applied in order and nesting to the +/// left, as [`NonEmpty::with`] nests: a field's `users/age` extended by `7` +/// and then `"eu"` ([`with`](Self::with) twice) is `((users/age)/7u64)/eu`, +/// the context a binding whose caller extends part by part spells. One +/// `NonEmpty` holding both parts is a different context, `(users/age)/(7u64/eu)`. #[derive(Clone, Debug, Default)] -pub struct DeclaredContext(Option); +pub struct DeclaredContext(Vec); impl From<()> for DeclaredContext { fn from(_: ()) -> Self { Self::default() @@ -148,23 +163,37 @@ impl From<()> for DeclaredContext { } impl From for DeclaredContext { fn from(value: CallerContext) -> Self { - Self(Some(value)) + Self(vec![value]) } } impl<'a, T: IntoContext<'a>> From> for DeclaredContext { fn from(value: NonEmpty) -> Self { - Self(Some(value.into())) + Self(vec![value.into()]) } } impl DeclaredContext { + /// Extend by one more part, after any already given: the field's own + /// context, extended by every part so far, is extended by `part`. + pub fn with(mut self, part: impl Into) -> Self { + self.0.push(part.into()); + self + } + /// The context one field is derived under: its own `own`, extended by /// the caller's context if one was given — `("users", "age")` as it is - /// under `()`, `(("users", "age"), id)` under a caller's `id`. + /// under `()`, `(("users", "age"), id)` under a caller's `id`, and + /// `((("users", "age"), id), region)` under `id` then `region`. pub fn under<'c, O: IntoContext<'c>>(self, own: NonEmpty) -> CallerContext { - match self.0 { - Some(caller) => caller.extend(own), - None => own.into(), - } + let mut parts = self.0.into_iter(); + let Some(first) = parts.next() else { + return own.into(); + }; + parts.fold(first.extend(own), |extended, part| { + // `extended` is a `CallerContext`, nonempty by construction, so + // extending it is `(extended, part)` at the piece level: the + // same tree `NonEmpty::with` builds. + CallerContext(ContextPiece::List(vec![extended.0, part.0])) + }) } } diff --git a/packages/stack-encrypt/src/target/index.rs b/packages/stack-encrypt/src/target/index.rs index 8f641f4dc..bbadf6ae2 100644 --- a/packages/stack-encrypt/src/target/index.rs +++ b/packages/stack-encrypt/src/target/index.rs @@ -426,6 +426,39 @@ mod tuples { } } +/// A set of indexes whose size is known only at run time: what a plan +/// lowered from data holds, where the indexes arrive as a list rather than +/// a tuple. The terms come out as a `Vec`, one per index, in order. +/// +/// A `Vec` cannot be refused for being empty at compile time as `()` is, so +/// an empty one is refused when it runs, with +/// [`PlanError::EmptyIndexes`](crate::PlanError::EmptyIndexes), before any +/// key is requested: a field declared indexed derives at least one term. +/// [`select`](Indexes::select) is not available on a `Vec`; a binding +/// answers a query by index key, through its own `Index` impl. +impl> Indexes for Vec { + type Terms = Vec; + fn specs(&self) -> Vec { + self.iter().map(Index::spec).collect() + } + fn operations<'s, K: 'static, M>(&self) -> Encryption<'s, S, Self::Terms, K, CallerContext, M> + where + S: 's, + M: ConsumeSource<'s, S> + ShareSource<'s, S>, + { + let mut indexes = self.iter(); + let Some(first) = indexes.next() else { + return Encryption::failed(crate::PlanError::EmptyIndexes.into()); + }; + indexes.fold(first.operation().map(|term| vec![term]), |terms, index| { + terms.zip(index.operation()).map(|(mut terms, term)| { + terms.push(term); + terms + }) + }) + } +} + /// One value, sealed, with the terms of its indexes beside it: what /// [`indexed`] produces. /// diff --git a/packages/stack-encrypt/src/target/mod.rs b/packages/stack-encrypt/src/target/mod.rs index b65cb29cf..565e370d1 100644 --- a/packages/stack-encrypt/src/target/mod.rs +++ b/packages/stack-encrypt/src/target/mod.rs @@ -101,6 +101,8 @@ pub use index::{ indexed, At, Encrypted, Equality, Index, IndexSpec, Indexes, Match, Ope, Ore, Select, TermSet, Whole, }; +#[cfg(feature = "dynamic")] +pub(crate) use operations::chosen; pub(crate) use operations::inspect; pub use operations::{ ciphertext, equality, matching, ope, open, ore, passthrough, DecryptField, DecryptFrom, diff --git a/packages/stack-encrypt/src/target/operations.rs b/packages/stack-encrypt/src/target/operations.rs index 604d3d46f..ce772f9a9 100644 --- a/packages/stack-encrypt/src/target/operations.rs +++ b/packages/stack-encrypt/src/target/operations.rs @@ -373,6 +373,32 @@ impl<'s, S: 's, T: 'static, K: 'static, Ctx: 's> Encryption<'s, S, T, K, Ctx> { } } +/// A description whose operation is chosen by the value it is handed, at +/// the moment it runs: `choose` sees the source, the keyset cipher and the +/// context, and returns the pending of whichever typed operation applies. +/// +/// Crate-internal, and deliberately so: this is the one constructor that +/// hands a closure the plaintext and the cipher together, which the public +/// constructors never do. Its one caller is the `dynamic` module, where a +/// value's type is known only at run time and the step that stays dynamic +/// is dispatching a runtime scalar to the typed term operation a Rust +/// caller would have named. Everything else about the field (its context, +/// its place in the record, its batching) is the engine's, as for any other +/// description. +#[cfg(feature = "dynamic")] +pub(crate) fn chosen<'s, S: 's, T: 'static, K: 'static, Ctx: 's, M: SourceMode<'s, S>, G>( + choose: G, +) -> Encryption<'s, S, T, K, Ctx, M> +where + G: for<'a, 'k> FnOnce(M::Source, &'a KeysetCipher<'k, K>, Ctx) -> Pending<'a, T, K> + + MaybeSend + + 's, +{ + Encryption { + build: Box::new(choose), + } +} + /// A description that only checks the borrowed source, yielding `()` or the /// error without I/O. Zipped beside a record's fields, a failed check fails /// the whole record before any key is requested. Crate-internal: the plan From c1a4f4ba3e77d827756066201e0ea6e4de695548 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 01:00:42 -0700 Subject: [PATCH 02/14] test(stack-encrypt): the record fixture is the proof of the lowering ADR-0007 as amended on 2026-10-06 makes a shared fixture, not a snapshot, the proof that the typed chain and the data-plan lowering are one engine: each opens the records the other sealed, and both derive the same bytes for each term. tests/fixtures/record_lowering.json holds one record from each author under one declaration, with their term bytes, and tests/record_lowering.rs opens each with the other, today's records and the committed ones alike. The README beside it gives the schema a Go test reads later, once generated Go code is the third author; the Go reader is not in this change. FakeDataKeySource hands out random keys and remembers them in-process, so a committed record sealed under it could never open again. The fixture is sealed under a DeterministicSource in tests/common instead: every key and tag is SHA-256 over a seed, the leaf's descriptor and its IV, so a reader with the seed re-derives the key from what the leaf stores, and a leaf moved under another field's label is refused as ZeroKMS would refuse it (a test moves one). The index key is the fake's, deterministic per keyset, so the fixture's terms are the terms every other test derives. sha2 joins the dev-dependencies for it, at the version stack-kms's fake already uses. Regenerate with STACK_ENCRYPT_UPDATE_FIXTURES=1. Only the sealed bytes change between runs, since every leaf carries a fresh nonce; the test fails if the plan, the plaintext or the terms do. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- Cargo.lock | 1 + packages/stack-encrypt/Cargo.toml | 5 + packages/stack-encrypt/tests/common/mod.rs | 111 ++++ .../stack-encrypt/tests/fixtures/README.md | 64 +++ .../tests/fixtures/record_lowering.json | 98 ++++ .../stack-encrypt/tests/record_lowering.rs | 500 ++++++++++++++++++ 6 files changed, 779 insertions(+) create mode 100644 packages/stack-encrypt/tests/fixtures/README.md create mode 100644 packages/stack-encrypt/tests/fixtures/record_lowering.json create mode 100644 packages/stack-encrypt/tests/record_lowering.rs diff --git a/Cargo.lock b/Cargo.lock index 60687ec98..94e2dcaeb 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3281,6 +3281,7 @@ dependencies = [ "cllw-ore", "serde", "serde_json", + "sha2 0.10.9", "stack-auth", "stack-encrypt-derive", "stack-kms", diff --git a/packages/stack-encrypt/Cargo.toml b/packages/stack-encrypt/Cargo.toml index ea9c805d6..64100be44 100644 --- a/packages/stack-encrypt/Cargo.toml +++ b/packages/stack-encrypt/Cargo.toml @@ -64,6 +64,11 @@ test-support = ["stack-kms/test-support"] [dev-dependencies] serde_json = { workspace = true } +# The deterministic key source the record fixture is sealed under +# (`tests/common`): keys and tags derived from a seed, the descriptor and the +# IV, so a committed record opens in another process. Same version stack-kms +# uses for its fake. +sha2 = "0.10.6" # `default-features = false` here too: dev-dependency features unify into the # `cargo test -p stack-encrypt --no-default-features` graph, so leaving the # default on would silently pull `stack-kms/http` -> `stack-auth/http` -> diff --git a/packages/stack-encrypt/tests/common/mod.rs b/packages/stack-encrypt/tests/common/mod.rs index 20fdd0fff..fcf2a984e 100644 --- a/packages/stack-encrypt/tests/common/mod.rs +++ b/packages/stack-encrypt/tests/common/mod.rs @@ -219,3 +219,114 @@ pub async fn recording_cipher() -> (StackCipher, Arc Self { + Self { + seed, + counter: std::sync::atomic::AtomicU64::new(0), + index: FakeDataKeySource::new(), + } + } + + fn derive(&self, what: &str, descriptor: &str, salt: &[u8]) -> [u8; 32] { + use sha2::{Digest, Sha256}; + let mut hasher = Sha256::new(); + hasher.update(self.seed); + hasher.update(what.as_bytes()); + hasher.update([0u8]); + hasher.update(descriptor.as_bytes()); + hasher.update([0u8]); + hasher.update(salt); + hasher.finalize().into() + } +} + +impl DataKeySource for DeterministicSource { + async fn generate_keys( + &self, + payloads: Vec>, + _keyset_id: Option, + _unverified_context: Option>, + ) -> Result, stack_kms::Error> { + Ok(payloads + .into_iter() + .map(|payload| { + let n = self.counter.fetch_add(1, AtomicOrdering::SeqCst); + let iv_bytes = self.derive("iv", payload.descriptor, &n.to_le_bytes()); + let mut iv = stack_kms::Iv::default(); + let width = iv.len(); + iv.copy_from_slice(&iv_bytes[..width]); + let key = self.derive("key", payload.descriptor, &iv); + let tag = self.derive("tag", payload.descriptor, &iv); + DataKeyWithTag { + key: DataKey { iv, key }, + tag: tag.to_vec(), + decryption_policy: payload.decryption_policy, + } + }) + .collect()) + } + + async fn retrieve_keys( + &self, + payloads: Vec>, + _keyset_id: Option, + _unverified_context: Option<&UnverifiedContext>, + ) -> Result, stack_kms::Error> { + payloads + .iter() + .map(|payload| { + let iv: stack_kms::Iv = *payload.iv.as_ref(); + let tag = self.derive("tag", payload.descriptor, &iv); + if tag[..] != *payload.tag { + return Err(stack_kms::Error::RetrieveKey( + stack_kms::RetrieveKeyError::FailedRetrieval( + "the tag is not this descriptor's".to_string(), + ), + )); + } + let key = self.derive("key", payload.descriptor, &iv); + Ok(DataKey { iv, key }) + }) + .collect() + } +} + +impl IndexKeySource for DeterministicSource { + async fn load_index_key( + &self, + keyset_id: Option, + ) -> Result<(Uuid, IndexKey), stack_kms::Error> { + self.index.load_index_key(keyset_id).await + } +} + +/// A cipher over [`DeterministicSource`] with `seed`. +pub async fn deterministic_cipher(seed: [u8; 32]) -> StackCipher { + StackCipher::builder() + .kms(DeterministicSource::new(seed)) + .init() + .await + .expect("build cipher") +} diff --git a/packages/stack-encrypt/tests/fixtures/README.md b/packages/stack-encrypt/tests/fixtures/README.md new file mode 100644 index 000000000..66f20a6e8 --- /dev/null +++ b/packages/stack-encrypt/tests/fixtures/README.md @@ -0,0 +1,64 @@ +# Record fixture: `record_lowering.json` + +One declaration, two authors, one engine. The typed Rust chain +(`cipher.encrypt(&user).context("users").fields()…`) and the data-plan +lowering (`stack_encrypt::dynamic::record`, what a language binding +calls) each sealed the same plaintext under the same declaration. The +fixture holds both records, with their term bytes, sealed under a +deterministic key source so they open in any process built from the same +seed. `tests/record_lowering.rs` opens each record with the other author +and checks that both derive the terms it holds. That cross-opening is the +proof that `dynamic::record` is a lowering into the plan builder and not a +second executor (ADR-0007, amended 2026-10-06). A Go test reads the same +file later, once the Go SDK's generated code is the third author. + +Regenerate with `STACK_ENCRYPT_UPDATE_FIXTURES=1 cargo test -p +stack-encrypt --all-features --test record_lowering`. Only the sealed bytes +change between runs (every leaf carries a fresh nonce); the plan, the +plaintext and the terms must not, and the test fails if they do. Do not edit +the file by hand. + +## Schema + +```json +{ + "_comment": "what this file is", + "key_source": { "kind": "deterministic-sha256", "seed": "<32 bytes, hex>" }, + "keyset_id": "", + "plan": { "": { "context": [...], "outputs": [...], "type": "" }, ... }, + "plaintext": { "": , ... }, + "records": { + "typed_chain": { "": { "": , ... }, ... }, + "lowering": { "": { "": , ... }, ... } + } +} +``` + +- **`key_source`.** The test double in `tests/common/mod.rs` + (`DeterministicSource`): every data key is `SHA-256(seed ‖ "key" ‖ 0 ‖ + descriptor ‖ 0 ‖ iv)`, its tag `SHA-256(seed ‖ "tag" ‖ 0 ‖ descriptor ‖ 0 ‖ + iv)`, and the IV `SHA-256(seed ‖ "iv" ‖ 0 ‖ descriptor ‖ 0 ‖ counter)[..16]`, + where `descriptor` is the context the leaf was sealed under, rendered as + ZeroKMS logs it (`users/age`). A reader that re-derives the key from a + leaf's IV and its field's descriptor can open the leaf; a leaf moved under + another field's label does not open, as it would not under ZeroKMS. The + index key is `FakeDataKeySource`'s: `SHA-256("stack-kms::FakeDataKeySource::index-key::v1" + ‖ keyset_id)`, the same every test in the repository derives terms under. +- **`plan`.** The declaration in the data grammar `dynamic::record::plan` + parses: per field its label (a list of plain segments), its outputs (`"c"`, + `"passthrough"`, or an index key `"eq"`, `"match"`, `"ore"`, `"ope"`) and its + `"type"` (a vitaminc `ValueKind` name). The typed chain writes the same + declaration as `Plan::context("users").fields()` with one verb per field. +- **`plaintext`.** The value both authors sealed, each field as JSON at the + kind `plan` declares for it. +- **`records`.** Each record in the stored shape, field by field, output by + output: `"c"` is the field's ciphertext as the frozen `SealedValue` leaf + bytes, hex (the storage encoding a database column holds, see + `tests/frozen_bytes.rs`); an index key is the term's frozen bytes, hex (an + equality term's raw 32 PRF bytes, an ORE term's raw CLLW bytes, a match + term's little-endian `u16` positions); `"passthrough"` is the value as it + is. The terms of the two records are identical; the ciphertexts are not + (fresh nonces) but open under the same keys. + +`label_segments.json` beside this file is a different fixture: the one rule +for a plain label segment, read by the Rust, derive and Go tests. diff --git a/packages/stack-encrypt/tests/fixtures/record_lowering.json b/packages/stack-encrypt/tests/fixtures/record_lowering.json new file mode 100644 index 000000000..331f04908 --- /dev/null +++ b/packages/stack-encrypt/tests/fixtures/record_lowering.json @@ -0,0 +1,98 @@ +{ + "_comment": "Records sealed by the typed Rust chain and by the data-plan lowering under one declaration (ADR-0007). Read by tests/record_lowering.rs; the schema is in README.md beside this file. Regenerate with STACK_ENCRYPT_UPDATE_FIXTURES=1; do not edit by hand.", + "key_source": { + "kind": "deterministic-sha256", + "seed": "737461636b2d656e6372797074207265636f7264206669787475726520763120" + }, + "keyset_id": "00000000-0000-0000-0000-000000000000", + "plaintext": { + "age": 34, + "email": "bob@example.com", + "id": 42, + "notes": "likes cats" + }, + "plan": { + "age": { + "context": [ + "users", + "age" + ], + "outputs": [ + "c", + "eq", + "ore" + ], + "type": "uint32" + }, + "email": { + "context": [ + "users", + "email" + ], + "outputs": [ + "c", + "eq", + "match" + ], + "type": "string" + }, + "id": { + "context": [ + "users", + "id" + ], + "outputs": [ + "passthrough" + ], + "type": "uint32" + }, + "notes": { + "context": [ + "users", + "notes" + ], + "outputs": [ + "c" + ], + "type": "string" + } + }, + "records": { + "lowering": { + "age": { + "c": "01000000000000000000000000000000004d5626148f77c4c6d9da0ddbdf7663a2200021162641133485216f389886cbd7c6e9b25afa0f5f166f985b23384e81ffcfac01e3f8f61316f918d46ed5980e418c10738cec2c74de854df7cf1d0c4cfdca1ce5", + "eq": "5433335e609685a33b29c98369e68cd5d92e838aa566a3ff1df5c2d251b63885", + "ore": "7ca54e16d7944c0a9d8577d87f3864fdf79e99fea5956ba956a126a605cbf3f1" + }, + "email": { + "c": "0100000000000000000000000000000000b2afa77357a61488146c0edaa0246f65200043956c4f4bba857028c28165dfce9c63d904b738bbb039b6e11c805112d4a1ee0121e959ee736d9143b3f96f6e9d98720cfdc9f143e9f2b2cac7b85dc791eab4f514d5053b85cfb2ec61836e", + "eq": "ec534cfbf2336785378c3761ff88623c46361b2ba7c5fa83930ebacae49cf646", + "match": "06000d0013001d002200260029002a003000320043005a006000660068006c0077008f00960098009e00c200c500d600d900de00e300e700e900eb00ec00f200fa00" + }, + "id": { + "passthrough": 42 + }, + "notes": { + "c": "0100000000000000000000000000000000723f7a7f14b07197168aa13bc52010b3200072a73194b45af604cc6201b5a57554b05cc7ed6ad22b9102cbcb71bd9067849601103f61ed1d241e16a6995f2af90fd47e5b61a68b3f9f750a1e902048648a746782626468d388" + } + }, + "typed_chain": { + "age": { + "c": "010000000000000000000000000000000092e427c5e176b60261add6ce80fd02df20008d231453c97fdec97a0b6559ac8f509129d20163f262d40ce5d77cf1ebe0cad501c86e4a687dd91b4b0ac07922ff9b6195e2b1096d3b35bbc4ca54de28160dc164", + "eq": "5433335e609685a33b29c98369e68cd5d92e838aa566a3ff1df5c2d251b63885", + "ore": "7ca54e16d7944c0a9d8577d87f3864fdf79e99fea5956ba956a126a605cbf3f1" + }, + "email": { + "c": "0100000000000000000000000000000000cf92191950409f9fe82a2b11888ffc1d2000ca4d5bbafbff919af4d3a95c7cc046d632fd4c28785b238f4b6f7ea3aa08482901159eb270f977ce050a03335f53e1375462e3c42ab9d650374d6e1362ef3c89f7f065e4d1a80bd4111e0378", + "eq": "ec534cfbf2336785378c3761ff88623c46361b2ba7c5fa83930ebacae49cf646", + "match": "06000d0013001d002200260029002a003000320043005a006000660068006c0077008f00960098009e00c200c500d600d900de00e300e700e900eb00ec00f200fa00" + }, + "id": { + "passthrough": 42 + }, + "notes": { + "c": "0100000000000000000000000000000000562a41e032316ff2d1264a9e9f5c3b842000de33c8d1f2caaab8b90f691161a57eeb1328f7f1c9b5be76dec83e8a53765f1101905680f4433c1de6c861814a0c40bb3cfc25a2f6fcbda1f82e75a4dae75f018e90aa16d294e9" + } + } + } +} diff --git a/packages/stack-encrypt/tests/record_lowering.rs b/packages/stack-encrypt/tests/record_lowering.rs new file mode 100644 index 000000000..3c3674809 --- /dev/null +++ b/packages/stack-encrypt/tests/record_lowering.rs @@ -0,0 +1,500 @@ +//! The record fixture is the proof of the lowering (ADR-0007, amended +//! 2026-10-06): the typed Rust chain and the data-plan lowering each open +//! the records the other sealed, and both derive the same bytes for each +//! term. `tests/fixtures/record_lowering.json` holds one record from each +//! author, sealed under a deterministic key source so the bytes open in any +//! process, and its README gives the schema a Go test reads later. +//! +//! Regenerate the fixture with `STACK_ENCRYPT_UPDATE_FIXTURES=1 cargo test +//! --test record_lowering --all-features`; every other run reads it. +#![cfg(feature = "dynamic")] + +mod common; + +use std::collections::BTreeMap; +use std::path::PathBuf; + +use common::{deterministic_cipher, DeterministicSource}; +use serde_json::{json, Map, Value as Json}; +use stack_encrypt::dynamic::record::{self, Plan as DataPlan}; +use stack_encrypt::dynamic::{FfiValue, Scope}; +use stack_encrypt::plan::{pick, FieldValues}; +use stack_encrypt::sem::{EqualityTerm, MatchTerms, OreTerm}; +use stack_encrypt::target::Encrypted; +use stack_encrypt::{ + CipherText, Equality, Match, Ore, Plan, SealedValue, StackCipher, StackCipherText, +}; +use vitaminc_protected::Controlled; + +const SEED: [u8; 32] = *b"stack-encrypt record fixture v1 "; + +struct User { + age: u32, + email: String, + notes: String, + id: u32, +} + +fn user() -> User { + User { + age: 34, + email: "bob@example.com".into(), + notes: "likes cats".into(), + id: 42, + } +} + +/// The one declaration, as the typed chain writes it. +fn typed_plan() -> Plan { + Plan::context("users") + .fields() + .encrypt_index(pick("age", |u: &User| &u.age), (Equality, Ore)) + .encrypt_index( + pick("email", |u: &User| &u.email), + (Equality, Match::default()), + ) + .encrypt(pick("notes", |u: &User| &u.notes)) + .passthrough(pick("id", |u: &User| &u.id)) + .build() + .expect("the typed plan builds") +} + +/// The same declaration as a binding sends it: the fixture's `plan`. +fn data_plan_json() -> Json { + json!({ + "age": { "context": ["users", "age"], "outputs": ["c", "eq", "ore"], "type": "uint32" }, + "email": { "context": ["users", "email"], "outputs": ["c", "eq", "match"], "type": "string" }, + "notes": { "context": ["users", "notes"], "outputs": ["c"], "type": "string" }, + "id": { "context": ["users", "id"], "outputs": ["passthrough"], "type": "uint32" }, + }) +} + +fn plaintext_json() -> Json { + let u = user(); + json!({ "age": u.age, "email": u.email, "notes": u.notes, "id": u.id }) +} + +// ---- JSON <-> FfiValue, for the subset the fixture uses -------------------- + +fn ffi_of_json(value: &Json) -> FfiValue { + match value { + Json::String(s) => FfiValue::String(s.as_str().into()), + Json::Array(items) => FfiValue::Array(items.iter().map(ffi_of_json).collect()), + Json::Object(entries) => FfiValue::Object( + entries + .iter() + .map(|(k, v)| (k.clone(), ffi_of_json(v))) + .collect(), + ), + Json::Number(n) => FfiValue::UInt64(n.as_u64().expect("a non-negative integer")), + other => panic!("the fixture does not use {other}"), + } +} + +/// The plaintext as the binding sends it: each field at the kind the plan +/// declares for it. +fn source_of(plaintext: &Json, plan: &DataPlan) -> FfiValue { + let fields = plaintext.as_object().expect("an object"); + FfiValue::Object( + plan.fields() + .iter() + .map(|field| { + let value = &fields[field.name()]; + let value = match field.field_type().expect("every fixture field is typed") { + stack_encrypt::dynamic::ValueKind::UInt32 => FfiValue::UInt32( + u32::try_from(value.as_u64().expect("an integer")).expect("fits a u32"), + ), + stack_encrypt::dynamic::ValueKind::String => { + FfiValue::String(value.as_str().expect("text").into()) + } + other => panic!("the fixture does not use {other}"), + }; + (field.name().to_string(), value) + }) + .collect(), + ) +} + +fn json_of_opened(value: FfiValue) -> Json { + match value { + FfiValue::Object(entries) => Json::Object( + entries + .into_iter() + .map(|(k, v)| (k, json_of_opened(v))) + .collect(), + ), + FfiValue::UInt32(v) => json!(v), + FfiValue::String(s) => { + Json::String(String::from_utf8(s.risky_ref().to_vec()).expect("utf8")) + } + _ => panic!("the fixture does not use this kind"), + } +} + +// ---- hex ---------------------------------------------------------------------- + +fn hex(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02x}")).collect() +} + +fn unhex(text: &str) -> Vec { + (0..text.len()) + .step_by(2) + .map(|i| u8::from_str_radix(&text[i..i + 2], 16).expect("hex")) + .collect() +} + +// ---- the stored record <-> JSON --------------------------------------------- + +/// A stored record (the wire tree the lowering writes) as JSON: `"c"` is the +/// `SealedValue` leaf's frozen bytes, a term is its bytes, a passthrough is +/// the value. +fn json_of_record(tree: StackCipherText) -> Json { + let CipherText::Map(fields) = tree else { + panic!("a record is a map of fields"); + }; + let mut out = Map::new(); + for (name, node) in fields { + let CipherText::Map(outputs) = node else { + panic!("a field is a map of outputs"); + }; + let mut field = Map::new(); + for (key, node) in outputs { + let value = match (key.as_str(), node) { + ("c", CipherText::Single(leaf)) => json!(hex(&leaf.to_bytes())), + ("passthrough", CipherText::Passthrough(payload)) => { + match *payload.downcast::().expect("a value") { + FfiValue::UInt32(v) => json!(v), + _ => panic!("the fixture's passthrough is a u32"), + } + } + (_, CipherText::Passthrough(payload)) => { + match *payload.downcast::().expect("a value") { + FfiValue::Bytes(bytes) => json!(hex(bytes.risky_ref())), + _ => panic!("a term is bytes"), + } + } + (key, _) => panic!("unexpected output {key}"), + }; + let _ = field.insert(key, value); + } + let _ = out.insert(name, Json::Object(field)); + } + Json::Object(out) +} + +fn record_of_json(record: &Json) -> StackCipherText { + let fields = record.as_object().expect("an object"); + CipherText::Map( + fields + .iter() + .map(|(name, outputs)| { + let outputs = outputs.as_object().expect("an object"); + let nodes = outputs + .iter() + .map(|(key, value)| { + let node = match key.as_str() { + "c" => CipherText::Single( + SealedValue::from_bytes(&unhex(value.as_str().expect("hex"))) + .expect("a frozen leaf"), + ), + "passthrough" => CipherText::Passthrough(Box::new(FfiValue::UInt32( + u32::try_from(value.as_u64().expect("an integer")).expect("u32"), + )) + as stack_encrypt::BoxedPassthrough), + _ => CipherText::Passthrough(Box::new(FfiValue::Bytes( + vitaminc_protected::Protected::new(unhex( + value.as_str().expect("hex"), + )), + )) + as stack_encrypt::BoxedPassthrough), + }; + (key.clone(), node) + }) + .collect(); + (name.clone(), CipherText::Map(nodes)) + }) + .collect(), + ) +} + +/// The typed chain's record, shaped as the stored tree, by hand: what a Go +/// program assembles from the engine's standard outputs. +fn record_of_typed(mut values: FieldValues) -> StackCipherText { + let age: Encrypted<(EqualityTerm, OreTerm)> = values.take("age").expect("age"); + let email: Encrypted<(EqualityTerm, MatchTerms)> = values.take("email").expect("email"); + let notes: StackCipherText = values.take("notes").expect("notes"); + let id: u32 = values.take("id").expect("id"); + let term = |bytes: Vec| -> StackCipherText { + CipherText::Passthrough( + Box::new(FfiValue::Bytes(vitaminc_protected::Protected::new(bytes))) + as stack_encrypt::BoxedPassthrough, + ) + }; + CipherText::Map(vec![ + ( + "age".into(), + CipherText::Map(vec![ + ("c".into(), age.ciphertext), + ("eq".into(), term(age.terms.0.to_bytes())), + ("ore".into(), term(age.terms.1.to_bytes())), + ]), + ), + ( + "email".into(), + CipherText::Map(vec![ + ("c".into(), email.ciphertext), + ("eq".into(), term(email.terms.0.to_bytes())), + ("match".into(), term(email.terms.1.to_bytes())), + ]), + ), + ("notes".into(), CipherText::Map(vec![("c".into(), notes)])), + ( + "id".into(), + CipherText::Map(vec![( + "passthrough".into(), + CipherText::Passthrough( + Box::new(FfiValue::UInt32(id)) as stack_encrypt::BoxedPassthrough + ), + )]), + ), + ]) +} + +/// A stored tree as the typed chain's record, for `open`: each sealed field +/// as its bare ciphertext, the passthrough as its value. +fn typed_of_record(tree: StackCipherText) -> FieldValues { + let CipherText::Map(fields) = tree else { + panic!("a record is a map"); + }; + let mut values = FieldValues::new(); + for (name, node) in fields { + let CipherText::Map(outputs) = node else { + panic!("a field is a map of outputs"); + }; + for (key, node) in outputs { + match key.as_str() { + "c" => { + let _ = values.insert(&name, node); + } + "passthrough" => { + let CipherText::Passthrough(payload) = node else { + panic!("a passthrough node"); + }; + let FfiValue::UInt32(id) = *payload.downcast::().expect("a value") + else { + panic!("a u32"); + }; + let _ = values.insert(&name, id); + } + _ => {} + } + } + } + values +} + +/// Every term of a record, by `field/key`, so two records' terms compare +/// whatever else differs. +fn terms_of(record: &Json) -> BTreeMap { + let mut terms = BTreeMap::new(); + for (name, outputs) in record.as_object().expect("an object") { + for (key, value) in outputs.as_object().expect("an object") { + if key != "c" && key != "passthrough" { + let _ = terms.insert( + format!("{name}/{key}"), + value.as_str().expect("hex").to_owned(), + ); + } + } + } + terms +} + +fn fixture_path() -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/record_lowering.json") +} + +async fn seal_typed(cipher: &StackCipher) -> StackCipherText { + let values = cipher + .encrypt(&user()) + .using(&typed_plan()) + .await + .expect("the typed chain seals"); + record_of_typed(values) +} + +async fn seal_lowered( + cipher: &StackCipher, + plan: &DataPlan, +) -> StackCipherText { + record::encrypt( + &cipher.default_keyset(), + source_of(&plaintext_json(), plan), + plan, + ) + .expect("the source fits") + .await + .expect("the lowering seals") +} + +async fn open_typed(cipher: &StackCipher, tree: StackCipherText) -> Json { + let mut opened = cipher + .open(typed_of_record(tree)) + .using(&typed_plan()) + .await + .expect("the typed chain opens it"); + json!({ + "age": opened.take::("age").expect("age"), + "email": opened.take::("email").expect("email"), + "notes": opened.take::("notes").expect("notes"), + "id": opened.take::("id").expect("id"), + }) +} + +async fn open_lowered( + cipher: &StackCipher, + tree: StackCipherText, + plan: &DataPlan, +) -> Json { + json_of_opened( + record::decrypt(Scope::Client(cipher), tree, plan) + .expect("the record fits") + .await + .expect("the lowering opens it"), + ) +} + +/// The typed chain and the lowering, run now under the same declaration: +/// the same terms, and each opens the other's record. +#[tokio::test] +async fn the_chain_and_the_lowering_are_one_engine_today() { + let cipher = deterministic_cipher(SEED).await; + let plan = record::plan(ffi_of_json(&data_plan_json())).expect("the data plan parses"); + + let typed = json_of_record(seal_typed(&cipher).await); + let lowered = json_of_record(seal_lowered(&cipher, &plan).await); + assert_eq!(terms_of(&typed), terms_of(&lowered), "the same terms"); + assert_eq!( + terms_of(&typed).len(), + 4, + "eq, ore, eq, match: every index derived" + ); + + assert_eq!( + open_lowered(&cipher, record_of_json(&typed), &plan).await, + plaintext_json(), + "the lowering opens the chain's record" + ); + assert_eq!( + open_typed(&cipher, record_of_json(&lowered)).await, + plaintext_json(), + "the chain opens the lowering's record" + ); +} + +/// The committed fixture: records each author sealed in an earlier run, +/// opened by the other now, their terms the ones derived now. With +/// `STACK_ENCRYPT_UPDATE_FIXTURES=1` the records are sealed afresh and +/// written; the terms and the plaintext never change. +#[tokio::test] +async fn the_committed_fixture_opens_both_ways() { + let cipher = deterministic_cipher(SEED).await; + let plan = record::plan(ffi_of_json(&data_plan_json())).expect("the data plan parses"); + let path = fixture_path(); + + if std::env::var_os("STACK_ENCRYPT_UPDATE_FIXTURES").is_some() { + let fixture = json!({ + "_comment": "Records sealed by the typed Rust chain and by the data-plan lowering under one declaration (ADR-0007). Read by tests/record_lowering.rs; the schema is in README.md beside this file. Regenerate with STACK_ENCRYPT_UPDATE_FIXTURES=1; do not edit by hand.", + "key_source": { "kind": "deterministic-sha256", "seed": hex(&SEED) }, + "keyset_id": cipher.default_keyset().keyset_id().to_string(), + "plan": data_plan_json(), + "plaintext": plaintext_json(), + "records": { + "typed_chain": json_of_record(seal_typed(&cipher).await), + "lowering": json_of_record(seal_lowered(&cipher, &plan).await), + }, + }); + let mut text = serde_json::to_string_pretty(&fixture).expect("serialise"); + text.push('\n'); + std::fs::write(&path, text).expect("write the fixture"); + } + + let fixture: Json = + serde_json::from_str(&std::fs::read_to_string(&path).expect("the fixture is committed")) + .expect("the fixture is JSON"); + assert_eq!(fixture["key_source"]["seed"], json!(hex(&SEED))); + assert_eq!( + fixture["keyset_id"], + json!(cipher.default_keyset().keyset_id().to_string()) + ); + assert_eq!( + fixture["plan"], + data_plan_json(), + "the fixture's declaration is this one" + ); + assert_eq!(fixture["plaintext"], plaintext_json()); + + let typed = &fixture["records"]["typed_chain"]; + let lowered = &fixture["records"]["lowering"]; + let now = json_of_record(seal_lowered(&cipher, &plan).await); + assert_eq!( + terms_of(typed), + terms_of(&now), + "the chain's fixture terms are today's" + ); + assert_eq!( + terms_of(lowered), + terms_of(&now), + "the lowering's fixture terms are today's" + ); + + assert_eq!( + open_lowered(&cipher, record_of_json(typed), &plan).await, + plaintext_json(), + "the lowering opens the chain's committed record" + ); + assert_eq!( + open_typed(&cipher, record_of_json(lowered)).await, + plaintext_json(), + "the chain opens the lowering's committed record" + ); + assert_eq!( + open_typed(&cipher, record_of_json(typed)).await, + plaintext_json(), + "and each opens its own" + ); + assert_eq!( + open_lowered(&cipher, record_of_json(lowered), &plan).await, + plaintext_json() + ); +} + +/// The deterministic source refuses a leaf opened under another descriptor, +/// as ZeroKMS does, so the fixture's records are bound to their labels and +/// not merely decodable. +#[tokio::test] +async fn the_fixture_records_are_bound_to_their_labels() { + let cipher = deterministic_cipher(SEED).await; + let plan = record::plan(ffi_of_json(&data_plan_json())).expect("plan"); + let sealed = seal_lowered(&cipher, &plan).await; + let CipherText::Map(mut fields) = sealed else { + panic!("a map"); + }; + // Swap the `age` and `notes` field names: each ciphertext now sits under + // the other's label. + for (name, _) in &mut fields { + *name = match name.as_str() { + "age" => "notes".into(), + "notes" => "age".into(), + other => other.into(), + }; + } + let result = record::decrypt(Scope::Client(&cipher), CipherText::Map(fields), &plan) + .expect("the shape fits") + .await; + assert!( + result.is_err(), + "a leaf under another field's label does not open" + ); +} From 2a400d0d4c5be4c9de67a6c09ae35b2cfe277280 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 01:07:15 -0700 Subject: [PATCH 03/14] fix(stack-encrypt): CI fixes for the lowering Two checks failed on the first push of this branch. The plan module's rustdoc linked `crate::dynamic::record`, which exists only with the `dynamic` feature, so `cargo doc --no-default-features` (the WASI job's no-http docs step) failed on a broken intra-doc link. The sentence now names the module without a link. Biome's formatter wanted the record fixture's JSON arrays on one line. The fixture is generated by the `record_lowering` test under `STACK_ENCRYPT_UPDATE_FIXTURES=1`, so it joins the other generated files that `biome.json` excludes rather than being hand-formatted after each regeneration. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- biome.json | 1 + packages/stack-encrypt/src/plan/mod.rs | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/biome.json b/biome.json index a70d1857d..099fd0918 100644 --- a/biome.json +++ b/biome.json @@ -10,6 +10,7 @@ "!languages/typescript/packages/protect-ffi/lib", "!languages/typescript/packages/protect-ffi/target", "!languages/typescript/packages/protect-ffi/src/eql-v3-types", + "!packages/stack-encrypt/tests/fixtures/record_lowering.json", "!packages/eql/crates/eql-bindings/bindings", "!packages/eql/crates/eql-bindings/schema", "!packages/eql/packages/eql/src/generated", diff --git a/packages/stack-encrypt/src/plan/mod.rs b/packages/stack-encrypt/src/plan/mod.rs index 9baa2ffbf..9b211fc50 100644 --- a/packages/stack-encrypt/src/plan/mod.rs +++ b/packages/stack-encrypt/src/plan/mod.rs @@ -262,7 +262,7 @@ //! # A binding lowers data into the same builder //! //! A language binding has no types to name, so it declares a record as -//! data. With the `dynamic` feature, [`dynamic::record`](crate::dynamic::record) +//! data. With the `dynamic` feature, the `dynamic::record` module //! reads that declaration and lowers it into this builder — one context, //! `encrypt` / `encrypt_index` / `index` / `passthrough` per field — and //! runs the result through [`KeysetCipher::run`](crate::KeysetCipher::run) From 0d26f0a08ec38e274ab77df3f50fc7ab5e8675c7 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 01:15:46 -0700 Subject: [PATCH 04/14] fix(go): a plan field's context is a label, refused at construction Codex, on PR #1093: the lowering refuses every plan field whose context is not a label of at least two plain segments, but NewPlan and PlanFromTags still accepted a one-part Context (MustContext("c"), the context= tag), a one-segment label, bytes or an integer, and Context.With on a field, so such a plan built fine and every record call then failed with ErrEncoding. The mistake belongs at the earliest stage (Go principle 1), so the Go side now refuses at plan construction what the lowering refuses. Context.fieldLabel is the rule, one place: a planned field's Context must be a flat list of plain text parts, two or more, and is read as that Label. A one-part context, an extended context and a part that is not text are refused with the field named and the accepted form in the message; a flat list built with NewContext("users").With("age") is the label it spells, as the lowering reads it. NewPlan applies it to every field; PlanFromTags refuses the context= tag outright, naming label= as the tag to use, after the option loop so a repeated or doubled option is still reported as what the author wrote. plan.Custom took one arbitrary text part and bound it with NewContext; it now parses its argument as a Label, so a Custom target binds a label of two or more plain segments and a one-segment or unplain one is refused at Build. plantest reads a Custom column's context the same way. The golden files do not change: a Custom context was always rendered as the text it was written as, which is a label's String() too. Two of the lowering's rules stay with the record call, documented on NewPlan: every field of a plan must sit under one table, and no two fields may bind one label. NewPlan cannot hold them without refusing policies the plan package and its golden tests pin (a Custom target beside a table's EQL columns; several fields under one Custom context), and that package is replaced by the next PR in the stack. Docs follow: Context, NewContext, MustContext, FieldPlan.Context, NewPlan, the stash tag table, Label's naming table and plan.Custom now say which contexts a planned field binds and which a probe takes. Tests cover each refusal; the tests that used the one-part form are rewritten to labels. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- languages/golang/stackencrypt/context.go | 51 +++++++++-- languages/golang/stackencrypt/label.go | 5 ++ languages/golang/stackencrypt/label_test.go | 25 ++++-- .../golang/stackencrypt/plan/plan_test.go | 18 ++-- .../stackencrypt/plan/plantest/snapshot.go | 15 ++-- languages/golang/stackencrypt/plan/policy.go | 19 ++-- languages/golang/stackencrypt/record.go | 70 ++++++++++----- languages/golang/stackencrypt/unit_test.go | 87 ++++++++++++++++--- 8 files changed, 221 insertions(+), 69 deletions(-) diff --git a/languages/golang/stackencrypt/context.go b/languages/golang/stackencrypt/context.go index f99cf6dab..27387f26f 100644 --- a/languages/golang/stackencrypt/context.go +++ b/languages/golang/stackencrypt/context.go @@ -16,15 +16,24 @@ import ( // A Context is a part or a list of parts. A part is a string, a byte slice // or an integer (int32, int64, uint32, uint64; Go's int is sent as int64). // [NewContext] makes a one-part context — the bare part, what a Rust -// `#[stash(context = "..")]` literal binds. [Context.With] extends it as +// `nonempty!("..")` literal binds. [Context.With] extends it as // Rust's NonEmpty::with does: the result is the two-element list // [previous, part], nesting to the left. So NewContext("users").With("age") // is the pair a Rust `struct = .., context = "users"` derive binds its `age` // field under — and what ParseLabel("users/age") binds — rendering the ZeroKMS // descriptor users/age; extended With(uint64(7)) it is what a row sealed -// with encrypt_into_with_context(row, 7u64) binds for that field. +// with the chain's .extend(7u64) binds for that field. // A one-element list is not the bare part, and this type cannot spell one. // +// Not every Context is a planned field's. A probe ([Cipher.Term]) takes any +// Context, in whatever shape the data was sealed under. A field of a record +// plan ([FieldPlan.Context]) binds a [Label] of at least two plain segments +// and nothing else — the guest lowers a plan into one context per record +// with one identity per field, and refuses any other shape — so [NewPlan] +// and [PlanFromTags] refuse a one-part context, a one-segment label and an +// extended context at construction. A record call extends every field's +// label alike with [ExtendContext]. +// // A Context owns its parts: a byte-slice part is copied in, so a caller's // buffer reused once the Context is built does not change it. // @@ -35,7 +44,11 @@ type Context struct { node any } -// NewContext makes a one-part context. The part must not be empty: a bare +// NewContext makes a one-part context, for a probe ([Cipher.Term]) against +// data sealed under one part, or as the base [Context.With] extends. It is +// not a planned field's context: a field binds a [Label] of two or more +// segments ([ParseLabel]), and [NewPlan] refuses a one-part context with the +// field named. The part must not be empty: a bare // empty string or empty byte slice is an empty context, and the guest // proves every context non-empty at the boundary, so such a Context could // only ever fail — every call, with ErrEncoding. Rust refuses the same @@ -56,8 +69,7 @@ func NewContext(part any) (Context, error) { } // MustContext is [NewContext] for a part known to be valid; it panics -// otherwise, an empty part included. For string literals in plans and -// probes. +// otherwise, an empty part included. For string literals in probes. func MustContext(part any) Context { c, err := NewContext(part) if err != nil { @@ -91,6 +103,35 @@ func ownPart(part any) any { // lists of scalars, ready for the transport codec. func (c Context) value() any { return c.node } +// fieldLabel is the context a planned field may bind, as the guest's +// lowering reads it: a label of at least two plain segments and nothing +// else, returned as that Label. A one-part context, an extended context and +// a part that is not text are refused with a reason that says what is +// accepted; a flat list of plain text parts is the label it spells, +// whichever constructor built it. +func (c Context) fieldLabel() (Label, error) { + parts, ok := c.node.([]any) + if !ok { + return Label{}, errors.New(`is one part, not a label; a planned field binds a label of at least two plain segments, ParseLabel("table/column").Context()`) + } + segments := make([]string, 0, len(parts)) + for _, part := range parts { + s, ok := part.(string) + if !ok { + return Label{}, errors.New("is extended, or holds a part that is not text; a planned field binds a plain label, and a record call extends every field's label alike with ExtendContext") + } + segments = append(segments, s) + } + if len(segments) < 2 { + return Label{}, errors.New("has one segment; a planned field binds a label of at least two") + } + l, err := NewLabel(segments...) + if err != nil { + return Label{}, fmt.Errorf("is not a plain label: %w", err) + } + return l, nil +} + // checkRootNonEmpty refuses the bare parts that are themselves an empty // context. Integers never are, whatever their value. func checkRootNonEmpty(part any) error { diff --git a/languages/golang/stackencrypt/label.go b/languages/golang/stackencrypt/label.go index 98ad01f0c..fe8dcf416 100644 --- a/languages/golang/stackencrypt/label.go +++ b/languages/golang/stackencrypt/label.go @@ -32,6 +32,11 @@ import ( // a deeper name ParseLabel("documents/v2/body") documents/v2/body // a one-part name ParseLabel("users"), the same as NewContext("users") users // +// A one-part name is a probe's: a planned field ([FieldPlan.Context]) binds +// a label of at least two segments, a table and a column, since the guest +// seals every field of a record under one context and the field's own +// identity. +// // Do not build a name with With, and do not put a scope into a Label. The // renderer keeps the two apart: a name is one flat list, a scope nests. So // (users/email)/7u64 is never read as a three-segment name, and diff --git a/languages/golang/stackencrypt/label_test.go b/languages/golang/stackencrypt/label_test.go index ae531c282..929e5342e 100644 --- a/languages/golang/stackencrypt/label_test.go +++ b/languages/golang/stackencrypt/label_test.go @@ -141,12 +141,13 @@ func label(t testing.TB, s string) Label { return l } -// A struct tag names a field's own context as a label or as one part, -// never both, and a plan built by hand needs a non-zero Context. -func TestTagsSpellALabelOrOneContextPart(t *testing.T) { +// A struct tag names a field's label, and only a label: the one-part +// context= form, which the guest's lowering cannot seal a field under, is +// refused with the tag to use instead, and a plan built by hand needs a +// non-zero Context. +func TestTagsSpellALabel(t *testing.T) { type tagged struct { Email string `stash:"label=users/email"` - Notes string `stash:"context=notes/v1"` } p, err := PlanFromTags(reflect.TypeOf(tagged{})) if err != nil { @@ -156,9 +157,19 @@ func TestTagsSpellALabelOrOneContextPart(t *testing.T) { if got := fields[0].Context.value(); !reflect.DeepEqual(got, []any{"users", "email"}) { t.Errorf("label=users/email bound %#v", got) } - // context= is one part: the '/' is text, as a Rust literal's is. - if got := fields[1].Context.value(); !reflect.DeepEqual(got, "notes/v1") { - t.Errorf("context=notes/v1 bound %#v, want the one part", got) + // context= is one part, and a planned field binds a label. + _, err = PlanFromTags(reflect.TypeOf(struct { + Notes string `stash:"context=notes/v1"` + }{})) + if err == nil || !strings.Contains(err.Error(), `context="notes/v1" names one text part`) || !strings.Contains(err.Error(), "use label=") { + t.Errorf("context=notes/v1: err = %v; want the one part refused and label= named", err) + } + // A one-segment label is one part too. + _, err = PlanFromTags(reflect.TypeOf(struct { + Notes string `stash:"label=notes"` + }{})) + if err == nil || !strings.Contains(err.Error(), "one part, not a label") { + t.Errorf("label=notes: err = %v; want the one segment refused", err) } for name, typ := range map[string]reflect.Type{ "both": reflect.TypeOf(struct { diff --git a/languages/golang/stackencrypt/plan/plan_test.go b/languages/golang/stackencrypt/plan/plan_test.go index f0cfaac29..1cdba0ce6 100644 --- a/languages/golang/stackencrypt/plan/plan_test.go +++ b/languages/golang/stackencrypt/plan/plan_test.go @@ -239,7 +239,7 @@ func TestContextsByTarget(t *testing.T) { want := []se.FieldPlan{ {Field: "Email", Name: "email", Context: label(t, "users/email").Context(), Terms: []se.TermKind{se.Equality}}, // A custom target's context is its own; the pin names the record key only. - {Field: "Blob", Name: "blob_v1", Context: se.MustContext("tenant-blobs/v1"), Terms: []se.TermKind{se.Ope}}, + {Field: "Blob", Name: "blob_v1", Context: label(t, "tenant-blobs/v1").Context(), Terms: []se.TermKind{se.Ope}}, } if got := p.Fields(); !reflect.DeepEqual(got, want) { t.Fatalf("fields =\n%+v\nwant\n%+v", got, want) @@ -275,7 +275,9 @@ func TestBuildRefusesMalformedDecisions(t *testing.T) { "nil target": {"t", plan.When(plan.Field("a"), plan.Encrypt(nil)), plan.ErrInvalid, "no target"}, "column on plain": {"t", plan.When(plan.Field("a"), plan.Plaintext(), plan.Column("c")), plan.ErrInvalid, "Plaintext"}, "identity on plain": {"t", plan.When(plan.Field("a"), plan.Plaintext(), plan.Identity("c")), plan.ErrInvalid, "Plaintext"}, - "identity on custom": {"t", plan.When(plan.Field("a"), plan.Encrypt(plan.Custom("ctx")), plan.Identity("c")), plan.ErrInvalid, "context is fixed"}, + "identity on custom": {"t", plan.When(plan.Field("a"), plan.Encrypt(plan.Custom("t/ctx")), plan.Identity("c")), plan.ErrInvalid, "context is fixed"}, + "one-part custom": {"t", plan.When(plan.Field("a"), plan.Encrypt(plan.Custom("ctx"))), nil, "one part, not a label"}, + "unplain custom": {"t", plan.When(plan.Field("a"), plan.Encrypt(plan.Custom("t/7up"))), plan.ErrInvalid, "is not a label"}, "slash in identity": {"t", plan.When(plan.Field("a"), plan.Encrypt(plan.EQL()), plan.Column("c"), plan.Identity("x/y")), plan.ErrInvalid, "contains '/'"}, // Identifier.Label() refuses more than '/': every reason a segment is // not plain, named as the table or the column identity it came from. @@ -326,8 +328,10 @@ func TestBuildRefusesMalformedDecisions(t *testing.T) { if !errors.Is(err, plan.ErrInvalid) || !strings.Contains(err.Error(), `identity "a" is already field a's`) { t.Errorf("two fields sharing an identity: err = %v", err) } - // Custom targets may share a context: it is the policy's to choose. - if _, err := plan.ForMessage(nil, "t", plan.When(plan.Any(plan.Field("a"), plan.Field("b")), plan.Encrypt(plan.Custom("ctx")))).Build(two); err != nil { + // Custom targets may share a context: it is the policy's to choose, and + // the guest, not NewPlan, refuses two fields under one label when a + // record call runs. + if _, err := plan.ForMessage(nil, "t", plan.When(plan.Any(plan.Field("a"), plan.Field("b")), plan.Encrypt(plan.Custom("blobs/ctx")))).Build(two); err != nil { t.Errorf("two custom fields sharing a context: %v", err) } } @@ -527,11 +531,11 @@ func TestABadIdentifierKeepsItsLabelError(t *testing.T) { // by such a field's context. func TestACustomOnlyMessageTakesAnyTableName(t *testing.T) { facts := []plan.Fact{{Field: "a", Annotations: []plan.Annotation{{Key: "k", Values: []string{"v"}}}}} - p, err := plan.ForMessage(nil, "a/b", plan.When(plan.Field("a"), plan.Encrypt(plan.Custom("ctx")))).Build(facts) + p, err := plan.ForMessage(nil, "a/b", plan.When(plan.Field("a"), plan.Encrypt(plan.Custom("blobs/ctx")))).Build(facts) if err != nil { t.Fatal(err) } - if got := p.Fields()[0].Context; !got.Equal(se.MustContext("ctx")) { - t.Errorf("context = %v, want the custom part", got) + if got := p.Fields()[0].Context; !got.Equal(label(t, "blobs/ctx").Context()) { + t.Errorf("context = %v, want the custom label", got) } } diff --git a/languages/golang/stackencrypt/plan/plantest/snapshot.go b/languages/golang/stackencrypt/plan/plantest/snapshot.go index ffb2ddd7d..3591dac79 100644 --- a/languages/golang/stackencrypt/plan/plantest/snapshot.go +++ b/languages/golang/stackencrypt/plan/plantest/snapshot.go @@ -121,9 +121,11 @@ func contextText(table string, kind string, d plan.Decision, fp stackencrypt.Fie if !ok || err != nil { return "", fmt.Errorf("plantest: column %q: cannot spell the context of target %v; a target that does not bind its column identity must be plan.Custom", fp.Name, target) } - if want, err = stackencrypt.NewContext(text); err != nil { + var l stackencrypt.Label + if l, err = stackencrypt.ParseLabel(text); err != nil { return "", fmt.Errorf("plantest: column %q: %w", fp.Name, err) } + want = l.Context() } if !want.Equal(fp.Context) { return "", fmt.Errorf("plantest: column %q: the plan binds a context other than %q", fp.Name, text) @@ -134,14 +136,11 @@ func contextText(table string, kind string, d plan.Decision, fp stackencrypt.Fie // contextOf is the context a snapshot's column names, rebuilt from its // text and target kind: what [contextText] wrote. func contextOf(c column) (stackencrypt.Context, error) { - if c.kind == kindEQL { - label, err := stackencrypt.ParseLabel(c.context) - if err != nil { - return stackencrypt.Context{}, err - } - return label.Context(), nil + label, err := stackencrypt.ParseLabel(c.context) + if err != nil { + return stackencrypt.Context{}, err } - return stackencrypt.NewContext(c.context) + return label.Context(), nil } // fact is one annotation value. diff --git a/languages/golang/stackencrypt/plan/policy.go b/languages/golang/stackencrypt/plan/policy.go index e77608d8f..6e8c8ac97 100644 --- a/languages/golang/stackencrypt/plan/policy.go +++ b/languages/golang/stackencrypt/plan/policy.go @@ -77,11 +77,11 @@ func (t eqlTarget) String() string { return "EQL(" + termList(t.terms) + ")" } // Custom is a non-EQL target: the field binds context, whatever its // column, and derives the given terms. The context is the policy's to // choose and, like any context, must never change once data is written -// under it. It is one arbitrary text part, exactly as written — what -// [stackencrypt.NewContext] makes and a Rust `#[stash(context = "..")]` -// literal binds — so a '/' in it is text, not a separator: "notes/v1" is -// one part, rendered escaped in the ZeroKMS log, never the table/column -// pair. A table and a column are an [EQL] target. +// under it. It is a label of at least two plain segments, written as +// [stackencrypt.ParseLabel] reads it ("notes/v1": the segments notes and +// v1, rendered as written in the ZeroKMS log), since that is the one shape +// of context a planned field binds; the message's table plays no part in +// it. A table and a column are an [EQL] target. func Custom(context string, terms ...stackencrypt.TermKind) Target { return customTarget{context: context, terms: slices.Clone(terms)} } @@ -93,7 +93,14 @@ type customTarget struct { func (t customTarget) Terms() []stackencrypt.TermKind { return slices.Clone(t.terms) } func (t customTarget) Context(Identifier) (stackencrypt.Context, error) { - return stackencrypt.NewContext(t.context) + if t.context == "" { + return stackencrypt.Context{}, errors.New("an empty string is an empty context") + } + l, err := stackencrypt.ParseLabel(t.context) + if err != nil { + return stackencrypt.Context{}, fmt.Errorf("context %q is not a label: %w", t.context, err) + } + return l.Context(), nil } func (t customTarget) String() string { return fmt.Sprintf("Custom(%q%s)", t.context, prefixed(termList(t.terms))) diff --git a/languages/golang/stackencrypt/record.go b/languages/golang/stackencrypt/record.go index 716a71a66..a8d882885 100644 --- a/languages/golang/stackencrypt/record.go +++ b/languages/golang/stackencrypt/record.go @@ -28,15 +28,16 @@ import ( // Notes string `stash:"label=users/notes"` // sealed only // } // -// Options are comma-separated: the field's own context as either -// `label=/` (a [Label], parsed with [ParseLabel]) or -// `context=` (one arbitrary text part, as [NewContext] makes it, what -// a Rust `#[stash(context = "..")]` literal binds) — exactly one of the two, -// required for a planned field; see [FieldPlan.Context] — `index=[;]` -// (eq, match, ore, ope), and `name=` (the record key; the Go -// field name otherwise). A field tagged `-` or `plain`, or not tagged at -// all, is not part of the record: it never crosses the boundary, and stays -// the caller's to store. Unexported fields are ignored. +// Options are comma-separated: the field's label, `label=
/` +// (a [Label] of at least two plain segments, parsed with [ParseLabel]), +// required for a planned field — see [FieldPlan.Context]; `index=[;]` +// (eq, match, ore, ope); and `name=` (the record key; the Go +// field name otherwise). The `context=` option, which named one text +// part as the field's whole context, is refused: the guest seals every +// field under the record's one context and the field's own identity, and a +// one-part context has neither. A field tagged `-` or `plain`, or not +// tagged at all, is not part of the record: it never crosses the boundary, +// and stays the caller's to store. Unexported fields are ignored. // // The same plan, built by hand. Each field's context is a [Label], the // table and the column; [ParseLabel] refuses a name that would not render @@ -68,9 +69,9 @@ import ( // // Every planned field is sealed (the `"c"` output). What the guest receives // is the same object whichever way the plan was built. The context each -// field binds is the plan's part, extended by [ExtendContext] parts exactly -// as the Rust derive extends a field's context by the caller's: -// NewContext(part).With(p1).With(p2). +// field binds is its label, extended by [ExtendContext] parts exactly as a +// Rust chain's .extend(..) extends a field's label: +// label.Context().With(p1).With(p2). // EncryptedField is one field's outputs from EncryptRecords: the sealed // ciphertext and whichever index terms the plan asked for (nil otherwise). @@ -210,17 +211,21 @@ type FieldPlan struct { // Name is the record key the field's outputs are stored under: the // column name, in EQL terms. Field when empty. Name string - // Context is the field's own encryption context; the record call - // extends it by any ExtendContext parts. Required. + // Context is the field's label: the context it is sealed and indexed + // under, before the record call extends it by any ExtendContext parts. + // Required. // - // A field stored in a database is named by a [Label] — a table and a + // It is a [Label] of at least two plain segments — a table and a // column, ParseLabel("users/age") then .Context(): the pair ["users", "age"] // a Rust `#[derive(EncryptFrom)]` with `struct = .., context = "users"` // binds its `age` field under, rendering the ZeroKMS descriptor - // users/age. Any other context is one [NewContext] makes: an arbitrary - // part, what a Rust `#[stash(context = "..")]` literal binds, rendered - // escaped if it would read as something else. A probe for the field - // ([Cipher.Term]) takes the same Context, so the two cannot drift. + // users/age. Nothing else: the guest lowers a record plan into one + // context per record (the label's leading segments) with one identity + // per field (its last), so a one-part context ([NewContext]), a + // one-segment label, a byte or integer part, and a Context already + // extended with [Context.With] are refused by [NewPlan], naming the + // field. A probe for the field ([Cipher.Term]) takes the same Context, + // so the two cannot drift. Context Context // Terms lists the terms to derive beside the ciphertext, in order. Terms []TermKind @@ -260,10 +265,18 @@ func (f planField) outputs() []string { } // NewPlan validates the fields and returns the plan. Every field needs a -// Field and a non-zero Context; Go field names must be unique, and so must record +// Field and a Context that is a label of at least two plain segments (see +// [FieldPlan.Context]); Go field names must be unique, and so must record // names (Name, or Field); Terms must be kinds this package defines, each -// at most once per field. A plan is built once and reused across calls, -// like the type it describes. +// at most once per field. These are the rules the guest holds a plan to +// when a record call runs, checked here so a mistake is found when the plan +// is built. Two rules stay with the call: every field of a plan must sit +// under one table (the label's leading segments), as a Rust chain's +// `Plan::context(table).fields()` does, and no two fields may bind one +// label, since they would share a context and their terms would be +// interchangeable; a plan that breaks either is refused by the record call +// that runs it. A plan is built once and reused across calls, like the type +// it describes. func NewPlan(fields ...FieldPlan) (Plan, error) { p, err := newPlan(fields) if err != nil { @@ -292,6 +305,9 @@ func newPlan(fields []FieldPlan) (Plan, error) { if f.Context.isZero() { return Plan{}, fmt.Errorf("plan field %s: a planned field needs a context", f.Field) } + if _, err := f.Context.fieldLabel(); err != nil { + return Plan{}, fmt.Errorf("plan field %s: context %w", f.Field, err) + } pf := planField{field: f.Field, name: f.Field, context: f.Context} if f.Name != "" { pf.name = f.Name @@ -354,7 +370,7 @@ func PlanFromTags(t reflect.Type) (Plan, error) { continue } pf := FieldPlan{Field: f.Name, Name: f.Name} - var ownKey string // the option that set pf.Context, for the message on a second one + var ownKey, ownValue string // the option that set pf.Context, for the message on a second one for _, opt := range strings.Split(tag, ",") { key, value, _ := strings.Cut(opt, "=") switch key { @@ -362,7 +378,7 @@ func PlanFromTags(t reflect.Type) (Plan, error) { if ownKey != "" { return Plan{}, fmt.Errorf("stackencrypt: field %s.%s: %s= and %s= both given; a field has one own context", t, f.Name, ownKey, key) } - ownKey = key + ownKey, ownValue = key, value var err error if key == "label" { var l Label @@ -392,6 +408,12 @@ func PlanFromTags(t reflect.Type) (Plan, error) { return Plan{}, fmt.Errorf("stackencrypt: field %s.%s: unknown stash tag option %q", t, f.Name, opt) } } + // Checked after the options so a repeated or doubled option is + // reported as what the author wrote; NewPlan would refuse the one + // part too, but this names the tag to use instead. + if ownKey == "context" { + return Plan{}, fmt.Errorf("stackencrypt: field %s.%s: context=%q names one text part, which a planned field cannot bind; use label=
/", t, f.Name, ownValue) + } fields = append(fields, pf) } if len(fields) == 0 { diff --git a/languages/golang/stackencrypt/unit_test.go b/languages/golang/stackencrypt/unit_test.go index 27f02d413..a8b9fa1e5 100644 --- a/languages/golang/stackencrypt/unit_test.go +++ b/languages/golang/stackencrypt/unit_test.go @@ -222,10 +222,10 @@ func TestPlanFromTags(t *testing.T) { A int `stash:"index=eq"` }{}, "unknown kind": struct { - A int `stash:"context=c,index=fuzzy"` + A int `stash:"label=t/c,index=fuzzy"` }{}, "unknown option": struct { - A int `stash:"context=c,store=true"` + A int `stash:"label=t/c,store=true"` }{}, "empty context": struct { A int `stash:"context="` @@ -234,11 +234,14 @@ func TestPlanFromTags(t *testing.T) { "not a struct": 42, "nil type": nil, "term twice": struct { - A int `stash:"context=c,index=eq;eq"` + A int `stash:"label=t/c,index=eq;eq"` }{}, "duplicate name": struct { - A int `stash:"context=c,name=x"` - B int `stash:"context=c,name=x"` + A int `stash:"label=t/c,name=x"` + B int `stash:"label=t/d,name=x"` + }{}, + "one-part context": struct { + A int `stash:"context=c"` }{}, } { if _, err := PlanFromTags(reflect.TypeOf(bad)); err == nil { @@ -462,7 +465,7 @@ func TestPlanBindsByFieldName(t *testing.T) { if _, err := PlanFromTags(typ); err == nil { t.Fatal("untagged struct has a tag plan") } - ok, err := NewPlan(FieldPlan{Field: "Email", Context: MustContext("c")}) + ok, err := NewPlan(FieldPlan{Field: "Email", Context: label(t, "t/c").Context()}) if err != nil { t.Fatal(err) } @@ -478,7 +481,7 @@ func TestPlanBindsByFieldName(t *testing.T) { "unexported": "hidden", "promoted": "Inner", } { - p, err := NewPlan(FieldPlan{Field: field, Context: MustContext("c")}) + p, err := NewPlan(FieldPlan{Field: field, Context: label(t, "t/c").Context()}) if err != nil { t.Fatal(err) } @@ -492,14 +495,16 @@ func TestPlanBindsByFieldName(t *testing.T) { } func TestNewPlanRefusesMalformedFields(t *testing.T) { + c := label(t, "t/c").Context() + d := label(t, "t/d").Context() for name, fields := range map[string][]FieldPlan{ "no fields": nil, - "no field name": {{Context: MustContext("c")}}, + "no field name": {{Context: c}}, "no context": {{Field: "A"}}, - "unknown kind": {{Field: "A", Context: MustContext("c"), Terms: []TermKind{TermKind(9)}}}, - "duplicate name": {{Field: "A", Context: MustContext("c"), Name: "x"}, {Field: "B", Context: MustContext("c"), Name: "x"}}, - "field twice": {{Field: "A", Name: "x", Context: MustContext("c")}, {Field: "A", Name: "y", Context: MustContext("d")}}, - "term twice": {{Field: "A", Context: MustContext("c"), Terms: []TermKind{Equality, Equality}}}, + "unknown kind": {{Field: "A", Context: c, Terms: []TermKind{TermKind(9)}}}, + "duplicate name": {{Field: "A", Context: c, Name: "x"}, {Field: "B", Context: d, Name: "x"}}, + "field twice": {{Field: "A", Name: "x", Context: c}, {Field: "A", Name: "y", Context: d}}, + "term twice": {{Field: "A", Context: c, Terms: []TermKind{Equality, Equality}}}, } { if _, err := NewPlan(fields...); err == nil { t.Errorf("%s: plan accepted", name) @@ -510,6 +515,64 @@ func TestNewPlanRefusesMalformedFields(t *testing.T) { } } +// A planned field's context is a label of at least two plain segments and +// nothing else: the one shape the guest lowers a plan field under. Every +// other Context the type can spell is refused when the plan is built, with +// the field named and the accepted form in the message, rather than by the +// guest at the first record call. +func TestNewPlanRefusesAContextThatIsNotAFieldLabel(t *testing.T) { + extended, err := label(t, "users/age").Context().With(uint64(7)) + if err != nil { + t.Fatal(err) + } + barePair, err := MustContext("c").With(uint64(7)) + if err != nil { + t.Fatal(err) + } + listPart, err := label(t, "users/age").Context().With("eu") + if err != nil { + t.Fatal(err) + } + for name, tc := range map[string]struct { + context Context + says string + }{ + "one text part": {MustContext("c"), "one part, not a label"}, + "one-segment label": {label(t, "users").Context(), "one part, not a label"}, + "an integer": {MustContext(uint64(7)), "one part, not a label"}, + "bytes": {MustContext([]byte{1, 2}), "one part, not a label"}, + "a label extended": {extended, "is extended"}, + "a bare part extended by an integer": {barePair, "is extended"}, + "a label extended by text": {listPart, "is extended"}, + } { + _, err := NewPlan(FieldPlan{Field: "Age", Context: tc.context}) + if err == nil { + t.Errorf("%s: NewPlan accepted it", name) + continue + } + if !strings.Contains(err.Error(), "plan field Age") || !strings.Contains(err.Error(), tc.says) { + t.Errorf("%s: err = %v; want the field named and %q", name, err, tc.says) + } + } + // A flat list of plain text parts is the label it spells, whichever + // constructor built it: the bytes are the label's. + pair, err := MustContext("users").With("age") + if err != nil { + t.Fatal(err) + } + if _, err := NewPlan(FieldPlan{Field: "Age", Context: pair}); err != nil { + t.Errorf("NewContext(\"users\").With(\"age\") refused: %v", err) + } + // A segment that is not plain is refused for the same reason a Label is. + if _, err := NewPlan(FieldPlan{Field: "Age", Context: Context{node: []any{"users", "7up"}}}); err == nil || !strings.Contains(err.Error(), "not a plain label") { + t.Errorf("a non-plain segment: err = %v", err) + } + // Validate on an explicit plan is NewPlan's check: the plan never built. + if _, err := NewPlan(FieldPlan{Field: "Age", Context: MustContext("c")}); err == nil { + t.Error("a one-part context built a plan Validate could be asked about") + } +} + func TestAssignFieldConvertsWithinFamiliesOnly(t *testing.T) { type flag bool type name string From d9c4ab5a7661b32b4ecfffd100af5cebaf8b5240 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 01:34:51 -0700 Subject: [PATCH 05/14] test(stack-encrypt): pin TermBytes's accessors and a Vec index set's specs The per-PR mutants gate reported seven survivors in the lowering's new code: TermBytes::as_bytes and its AsRef impl could return an empty slice or a one-byte one, and Indexes for Vec::specs could return an empty list, with no test the wiser. as_bytes and AsRef are the read side of a dynamic term, the pair every other term type offers, so they stay and a test now reads them: a term derived through IndexSpec's Index and through its Index dispatch is, through as_bytes, as_ref and into_bytes alike, the 32 PRF bytes the typed EqualityTerm holds. A Vec of indexes reports exactly its indexes' specs in order (one, two, a repeated one), derives its terms in that order byte for byte the tuple's, and an empty one is refused when it runs with PlanError::EmptyIndexes before any key request, which the earlier tests did not reach. cargo mutants -p stack-encrypt, filtered to the three functions: 9 mutants, 8 caught, 1 unviable. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- packages/stack-encrypt/src/dynamic/term.rs | 44 +++++++++++++ packages/stack-encrypt/tests/index.rs | 75 ++++++++++++++++++++++ 2 files changed, 119 insertions(+) diff --git a/packages/stack-encrypt/src/dynamic/term.rs b/packages/stack-encrypt/src/dynamic/term.rs index 1eecc49b1..cf3d17017 100644 --- a/packages/stack-encrypt/src/dynamic/term.rs +++ b/packages/stack-encrypt/src/dynamic/term.rs @@ -610,6 +610,50 @@ mod tests { .expect("build cipher") } + /// A `TermBytes` reads as the typed term's bytes, through every + /// accessor: `as_bytes`, `as_ref` and `into_bytes` give the same + /// 32 PRF bytes an `EqualityTerm` holds, for a term derived through + /// `IndexSpec`'s `Index` and through the dynamic dispatch alike. + #[tokio::test] + async fn term_bytes_read_as_the_typed_terms_bytes() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let typed = keyset + .equality_term(34u32, nonempty!("users/age")) + .await + .expect("typed") + .into_bytes() + .to_vec(); + assert_eq!(typed.len(), 32); + + let through_u32: TermBytes = keyset + .run( + Index::::operation::( + &IndexSpec::Equality, + ), + 34u32, + CallerContext::from(nonempty!("users/age")), + ) + .await + .expect("the u32 index derives"); + assert_eq!(through_u32.as_bytes(), typed.as_slice()); + assert_eq!(through_u32.as_ref(), typed.as_slice()); + assert_eq!(through_u32.clone().into_bytes(), typed); + + let through_value: TermBytes = keyset + .run( + Index::::operation::( + &IndexSpec::Equality, + ), + Value::new(FfiValue::UInt32(34)), + CallerContext::from(nonempty!("users/age")), + ) + .await + .expect("the dynamic index derives"); + assert_eq!(through_value, through_u32, "one dispatch table, one term"); + assert_eq!(through_value.as_bytes(), typed.as_slice()); + } + /// A match index under the default options: what a plan's bare /// `"match"` names. fn default_match() -> IndexSpec { diff --git a/packages/stack-encrypt/tests/index.rs b/packages/stack-encrypt/tests/index.rs index 0489dda12..82f1a68cc 100644 --- a/packages/stack-encrypt/tests/index.rs +++ b/packages/stack-encrypt/tests/index.rs @@ -27,6 +27,81 @@ use vitaminc_protected::{Controlled, Protected}; fn caller() -> CallerContext { CallerContext::from(nonempty!("users/email")) } + +/// A `Vec` of indexes is an index set sized at run time: its specs are the +/// indexes' in order, its terms a `Vec` in the same order, byte for byte the +/// tuple's; and an empty one is refused when it runs, before any key +/// request, since a field declared indexed derives at least one term. +#[tokio::test] +async fn a_vec_of_indexes_is_an_index_set_in_order_and_never_empty() { + let (cipher, generates, _) = counting_cipher().await; + let keyset = cipher.default_keyset(); + + let list: Vec = vec![IndexSpec::Ore, IndexSpec::Equality]; + assert_eq!( + Indexes::::specs(&list), + [IndexSpec::Ore, IndexSpec::Equality], + "the specs are the indexes', in the order given" + ); + assert_eq!( + Indexes::::specs(&vec![IndexSpec::Equality]), + [IndexSpec::Equality] + ); + assert_eq!( + Indexes::::specs(&vec![Equality, Equality]), + [IndexSpec::Equality, IndexSpec::Equality], + "a repeated index is reported twice; a plan refuses it, a set does not" + ); + + let terms: Vec = keyset + .run( + Indexes::::operations::<_, Borrowed>(&list), + &34u32, + caller(), + ) + .await + .unwrap(); + let (ore, eq): (OreTerm, EqualityTerm) = keyset + .run( + Indexes::::operations::<_, Borrowed>(&(Ore, Equality)), + &34u32, + caller(), + ) + .await + .unwrap(); + assert_eq!(terms.len(), 2); + assert_eq!( + terms[0].as_bytes(), + ore.as_bytes(), + "the first term is the first index's" + ); + assert_eq!( + terms[1].as_bytes(), + eq.as_bytes(), + "the second the second's" + ); + + let none: Vec = Vec::new(); + let refused: Result, _> = keyset + .run( + Indexes::::operations::<_, Borrowed>(&none), + &34u32, + caller(), + ) + .await; + assert!( + matches!( + refused, + Err(Error::Plan(stack_encrypt::PlanError::EmptyIndexes)) + ), + "{refused:?}" + ); + assert_eq!( + generates.load(Ordering::SeqCst), + 0, + "no key request either way" + ); +} fn aead() -> AeadContext { AeadContext::from(nonempty!("users/email")) } From 4ede502b48171dd60549d0d45b72dad8489df67e Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 01:38:17 -0700 Subject: [PATCH 06/14] test(stack-encrypt): gate the Vec index-set test on the dynamic feature `IndexSpec: Index` exists only with the `dynamic` feature, so the no-default-features test build in the WASI job failed to compile the new `tests/index.rs` case. The test now runs only when the feature is on, which is the only build that has a `Vec` index set to exercise. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- packages/stack-encrypt/tests/index.rs | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/packages/stack-encrypt/tests/index.rs b/packages/stack-encrypt/tests/index.rs index 82f1a68cc..88939d992 100644 --- a/packages/stack-encrypt/tests/index.rs +++ b/packages/stack-encrypt/tests/index.rs @@ -32,6 +32,10 @@ fn caller() -> CallerContext { /// indexes' in order, its terms a `Vec` in the same order, byte for byte the /// tuple's; and an empty one is refused when it runs, before any key /// request, since a field declared indexed derives at least one term. +/// `IndexSpec` implements `Index` only with the `dynamic` feature, so this +/// test is gated on it; the no-default-features build has no `Vec` index set +/// to exercise. +#[cfg(feature = "dynamic")] #[tokio::test] async fn a_vec_of_indexes_is_an_index_set_in_order_and_never_empty() { let (cipher, generates, _) = counting_cipher().await; From ad827c9e964f15fc804ec7d350f221b953b8101e Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 02:17:04 -0700 Subject: [PATCH 07/14] fix(stack-encrypt)!: a data plan field seals one leaf encoding, whatever its type cipherstash-bot, on PR #1093: a row sealed with no "type" opened under a plan that later declared "type": "string" as the same text with a leading U+000A, and no error. The typed string field sealed a bare String leaf and the untyped one vitaminc's tagged leaf, 0x0A then the UTF-8; a bare String reader accepts the tag as a line feed, and the opened value is still a string, so the kind check passed too. A uint32 field failed instead, with Error::Aead, which the guest reports as tampering. The two encodings cannot be told apart by inspection: a bare string that begins with U+000A is a valid tagged string, and a tagged string is a valid bare one. So no reader can refuse the other's leaf, and a type declared after rows exist was a silent migration. The lowering now seals every field as a Value, the tagged leaf, whatever its declared type. The type admits indexes and checks kinds, as before, and decides nothing about the bytes, so declaring one later changes no leaf and a binding that starts sending "type" (#1082) re-encrypts nothing; the tests pin both directions. The typed-leaf lowering, Leaf and the Index and Index impls of IndexSpec go with it; a lone IndexSpec is an index set of one, so a Rust chain over Value fields names its indexes as the data plan does. What this gives up, and where it is recorded: a Rust u32 or String field under the same label shares a data field's terms and not its leaf. A Rust record whose rows a binding must open declares Value fields, which is the same declaration as the data plan's, and the fixture now proves the lowering against such a chain. One leaf encoding both authors read, or the encoding bound into the leaf's context so the wrong reader fails closed, is a change to the Rust chain's bytes and is left to #1082 as a decision, named in the module docs and the CHANGELOG. The String-plaintext custody finding (record.rs:1092) is moot by the same change: no plaintext leaves its Protected buffer. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- packages/stack-encrypt/CHANGELOG.md | 39 +- packages/stack-encrypt/src/dynamic/mod.rs | 14 +- packages/stack-encrypt/src/dynamic/record.rs | 427 +++++++++--------- packages/stack-encrypt/src/dynamic/term.rs | 94 +--- packages/stack-encrypt/src/dynamic/value.rs | 14 +- packages/stack-encrypt/src/target/index.rs | 2 +- .../stack-encrypt/tests/fixtures/README.md | 31 +- .../tests/fixtures/record_lowering.json | 12 +- packages/stack-encrypt/tests/index.rs | 10 +- .../stack-encrypt/tests/record_lowering.rs | 72 +-- 10 files changed, 335 insertions(+), 380 deletions(-) diff --git a/packages/stack-encrypt/CHANGELOG.md b/packages/stack-encrypt/CHANGELOG.md index bf6828727..c5156ade0 100644 --- a/packages/stack-encrypt/CHANGELOG.md +++ b/packages/stack-encrypt/CHANGELOG.md @@ -49,14 +49,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 builder refuses them (`PlanError::SharedIdentity`). A record a Go program wrote with a `label=` tag is unaffected; one written with a `context=` tag is not readable through a plan. -- **A data plan field typed `uint32` or `string` seals as a bare `u32` or - `String` leaf** — the encoding a Rust `u32` or `String` field seals, run - through the same operations — so a derived record and a data plan - interchange ciphertexts and terms under one declaration. Every other kind, - and a field with no `"type"`, seals the tagged `FfiValue` encoding, as - before. A `uint32` or `string` field written by the previous release is - therefore not readable under a plan that now declares that type; written - without a type, it reads as before. +- **Every data plan field seals the tagged `FfiValue` leaf, whatever its + `"type"`**, as every field did before; the type admits indexes and checks + kinds and changes no bytes, so a row written without a type opens under a + plan that declares one, and a binding that starts sending `"type"` + re-encrypts nothing. A Rust `u32` or `String` field under the same label + derives the same terms as the data field but a different leaf, and the + two leaves cannot be told apart by inspection (a bare string that begins + with U+000A is a valid tagged string), so the lowering does not choose an + encoding from the type and a Rust record whose rows a binding must open + declares `dynamic::Value` fields, which are the same declaration as a data + plan's. One leaf encoding for both authors, or the encoding bound into + the leaf's context so the wrong reader fails closed, is a change to the + Rust chain's bytes and is left to #1082. - `dynamic::Output` gains `Passthrough` (the wire key `"passthrough"`). The enum is exhaustive on purpose, so a match over it must name the variant. - `target::DeclaredContext` holds several extension parts: `with(part)` @@ -138,17 +143,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `dynamic::Value`: an `FfiValue` as a plan field's plaintext, `Clone` by deep copy into fresh `Protected` payloads, so the borrowed engine can consume it. `dynamic::TermBytes`: a term as its frozen bytes. -- `IndexSpec` implements `Index`, `Index` and `Index`, - each with `TermBytes` as its term, so an index named as data runs through - `indexed()` like an index named as a type. `Vec` implements +- `IndexSpec` implements `Index`, with `TermBytes` as its term, so + an index named as data runs through `indexed()` like an index named as a + type, from a data plan or from a Rust chain over `Value` fields. `Vec` + implements `Indexes` for any `I: Index`: an index set sized at run time, whose terms are a `Vec`; an empty one is refused when it runs (`PlanError::EmptyIndexes`). -- `tests/fixtures/record_lowering.json`: records the typed chain and the - data-plan lowering each sealed under one declaration, with their term - bytes, under a deterministic key source; `tests/record_lowering.rs` opens - each with the other. The fixture is the proof that the two are one engine - (ADR-0007), and a Go test can read it later. +- `tests/fixtures/record_lowering.json`: records a Rust chain over `Value` + fields and the data-plan lowering each sealed under one declaration, with + their term bytes, under a deterministic key source; + `tests/record_lowering.rs` opens each with the other. The fixture is the + proof that the two are one engine (ADR-0007), and a Go test can read it + later. - A dynamic plan field may declare its type, as `"type": ""`. The vocabulary is vitaminc's `ValueKind` (re-exported as `dynamic::ValueKind`), not a new enum, so this crate now needs vitaminc diff --git a/packages/stack-encrypt/src/dynamic/mod.rs b/packages/stack-encrypt/src/dynamic/mod.rs index f53246438..04ceab49e 100644 --- a/packages/stack-encrypt/src/dynamic/mod.rs +++ b/packages/stack-encrypt/src/dynamic/mod.rs @@ -26,9 +26,9 @@ //! is the same bytes (ADR-0007). The one step that stays dynamic is //! dispatching a value whose type is known only at run time to the typed //! term operation, which [`IndexSpec`]'s `Index` impls do. -//! * [`Value`] — an [`FfiValue`] as a plan field's plaintext, where its -//! declared type names no Rust leaf type; [`TermBytes`] — the term such a -//! field derives, as its frozen bytes. +//! * [`Value`] — an [`FfiValue`] as a plan field's plaintext, the type of +//! every field lowered from data; [`TermBytes`] — the term such a field +//! derives, as its frozen bytes. //! * [`Scope`] — which cipher an opening operation decrypts through. //! //! # What is not here @@ -54,10 +54,10 @@ //! the type it was sealed as. They are not this crate's: a declared type is //! vitaminc's [`ValueKind`], re-exported here, whose names vitaminc freezes //! beside its tag table. This crate adds only what a kind means to an index -//! ([`admits`]) and to a query value ([`read`]), and which Rust leaf type a -//! field lowers to ([`record`]). A field without `"type"` is dispatched on -//! each value's own tag; that is transitional, and [`record::plan`] says -//! until when. +//! ([`admits`]) and to a query value ([`read`]); it decides nothing about +//! the bytes ([`record`]). A field without `"type"` is dispatched on each +//! value's own tag; that is transitional, and [`record::plan`] says until +//! when. //! //! For the same reason the enums that spell them — [`Output`], //! [`IndexSpec`] and [`ValueKind`] — are *not* `#[non_exhaustive]`, against this workspace's diff --git a/packages/stack-encrypt/src/dynamic/record.rs b/packages/stack-encrypt/src/dynamic/record.rs index 9e523e01b..02de226b7 100644 --- a/packages/stack-encrypt/src/dynamic/record.rs +++ b/packages/stack-encrypt/src/dynamic/record.rs @@ -15,7 +15,7 @@ //! //! What stays dynamic is one step: a field whose type is known only when its //! value arrives dispatches to the typed term operation then, through -//! [`IndexSpec`]'s [`Index`] impls (`scalar_term` in the `term` module is the +//! [`IndexSpec`]'s [`Index`](crate::target::Index) impl (`scalar_term` in the `term` module is the //! one table). //! //! # What a field's `"context"` must be @@ -33,20 +33,36 @@ //! an integer, bytes, a one-element list) is refused as a plan a fields plan //! cannot express. //! -//! # What a field's `"type"` decides +//! # What a field's `"type"` decides, and what it does not //! -//! The declared type is the data form of the Rust chain's `::`. A field -//! typed `uint32` or `string` lowers to a `u32` or `String` field — read out -//! of the value as that type, sealed and indexed through exactly the -//! operations `encrypt_index::` runs — so a `uint32` field of a Go -//! record and a `u32` field of a Rust record interchange, ciphertext and -//! terms alike. Every other kind, and a field with no `"type"`, is a -//! [`Value`]: it seals in vitaminc's self-describing tagged leaf encoding -//! (`[tag] ++ payload`), which only a dynamic reader opens, and its terms -//! dispatch on each value's own variant. The kinds with no bare Rust leaf -//! type (`uint64`, `int32`, `bool`, the floats, …) have no other encoding to -//! take; a field with no type has no type to name, which is transitional -//! (#1082). +//! Every field lowered from data is a [`Value`]: its plaintext type is the +//! runtime value itself, and it seals in vitaminc's self-describing tagged +//! leaf encoding (`[tag] ++ payload`) whatever its declared `"type"`. The +//! type decides which indexes the field admits ([`admits`]), which values it +//! seals and which it opens to (checked by kind, both ways), and nothing +//! about the bytes: declaring a type on a field written without one changes +//! no leaf, so a binding that starts sending `"type"` (#1082) re-encrypts +//! nothing. A field's terms dispatch on each value's own variant to the +//! typed term operation, so they are the bytes a Rust `u32` or `String` +//! field derives under the same label. +//! +//! The ciphertext is where a data plan and a Rust chain over bare types part: +//! a Rust `u32` field seals four bare bytes and a `String` field its bare +//! UTF-8, a `Value` seals the tagged leaf, and the two encodings cannot be +//! told apart by inspection — a bare string that begins with U+000A is a +//! valid tagged string, and a tagged string is a valid bare one with a line +//! feed in front — so neither reader can refuse the other's leaf and a +//! mis-declared one may read as changed plaintext rather than fail. The +//! lowering therefore never chooses an encoding from the type. A Rust record +//! whose rows a binding must open declares its fields as [`Value`] +//! (`encrypt_index(pick("age", |u: &User| &u.age), (IndexSpec::Equality, +//! IndexSpec::Ore))` over a `Value` field is the same declaration as the +//! data plan's, and the two interchange, leaves included — see +//! `tests/record_lowering.rs`); a Rust `u32` field shares a data field's +//! terms and nothing else. Closing that gap — one leaf encoding both authors +//! read, or the encoding bound into the leaf's context so the wrong reader +//! fails closed — is a change to the Rust chain's bytes, and so a decision +//! recorded against #1082, not something the lowering can take on its own. //! //! # The stored record is wire format //! @@ -66,11 +82,10 @@ //! that a round-trip invariant rather than data loss. use vitaminc_aead_value::{FfiValue, ValueKind}; -use vitaminc_protected::Controlled; use super::{admits, utf8, Error, Scalar, Scope, TermBytes, Value}; use crate::plan::{FieldValues, FieldsBuilder, Opens, Runs}; -use crate::target::{CallerContext, DeclaredContext, Decryption, Encrypted, Index, IndexSpec}; +use crate::target::{CallerContext, DeclaredContext, Decryption, Encrypted, IndexSpec}; use crate::{ BoxedPassthrough, CipherText, ContextPiece, KeysetCipher, Label, NonEmpty, Pending, StackCipherText, @@ -140,27 +155,6 @@ enum Verb { Passthrough, } -/// The Rust plaintext type a field lowers to, from its declared kind. -#[derive(Clone, Copy, PartialEq, Eq, Debug)] -enum Leaf { - /// `"uint32"`: a `u32`, as a Rust chain's `encrypt_index::` field. - U32, - /// `"string"`: a `String`. - Text, - /// Any other kind, or none: a [`Value`], the tagged leaf encoding. - Value, -} - -impl Leaf { - fn of(kind: Option) -> Self { - match kind { - Some(ValueKind::UInt32) => Leaf::U32, - Some(ValueKind::String) => Leaf::Text, - _ => Leaf::Value, - } - } -} - /// One field of a record plan: what to call it, what label to seal it /// under, what to produce for it, and, optionally, what type its values are. /// @@ -311,10 +305,6 @@ impl FieldPlan { } } - fn leaf(&self) -> Leaf { - Leaf::of(self.field_type) - } - /// The indexes the field declares, in output order. fn indexes(&self) -> Vec { self.outputs @@ -342,7 +332,6 @@ impl FieldPlan { FieldShape { name: self.name.clone(), verb: self.verb(), - leaf: self.leaf(), keys: self.indexes().iter().map(IndexSpec::key).collect(), kind: self.field_type, } @@ -461,17 +450,13 @@ impl Plan { /// The fields plan this declaration lowers to, for a cipher over `K`. /// /// Built afresh per call: a built plan is bound to its key source type, - /// and a declaration is not. Every field is declared by name, read out of - /// the [`FieldValues`] the source is converted into, at the Rust type its - /// kind lowers to. + /// and a declaration is not. Every field is declared by name, as a + /// [`Value`], read out of the [`FieldValues`] the source is converted + /// into. fn lower(&self) -> Result, crate::Error> { let mut builder = crate::Plan::context(self.context.clone()).fields::(); for field in &self.fields { - builder = match field.leaf() { - Leaf::U32 => declare::(builder, field), - Leaf::Text => declare::(builder, field), - Leaf::Value => declare::(builder, field), - }; + builder = declare(builder, field); if field.identity() != field.name { builder = builder.identity(field.identity()); } @@ -495,21 +480,18 @@ impl Plan { } } -/// One field's verb, at the type its kind lowers to. -fn declare( +/// One field's verb, over a [`Value`]; its indexes are the [`IndexSpec`]s +/// the plan named, each an [`Index`] of a `Value`. +fn declare( builder: FieldsBuilder, field: &FieldPlan, -) -> FieldsBuilder -where - F: crate::Encrypt + crate::Decrypt<'static> + Clone + Send + 'static, - IndexSpec: Index, -{ +) -> FieldsBuilder { let name = field.name.as_str(); match field.verb() { - Verb::Encrypt => builder.encrypt::(name), - Verb::EncryptIndex => builder.encrypt_index::(name, field.indexes()), - Verb::Index => builder.index::(name, field.indexes()), - Verb::Passthrough => builder.passthrough::(name), + Verb::Encrypt => builder.encrypt::(name), + Verb::EncryptIndex => builder.encrypt_index::(name, field.indexes()), + Verb::Index => builder.index::(name, field.indexes()), + Verb::Passthrough => builder.passthrough::(name), } } @@ -1043,7 +1025,7 @@ fn source_row(mut row: Vec<(String, FfiValue)>, plan: &Plan) -> Result Result<(), Error> { Ok(()) } -/// Put `value` in the record at the type `leaf` names. A value of another -/// kind than the leaf's — checked before this is reached on both paths — is -/// `misfit`. -fn insert_leaf( - values: &mut FieldValues, - name: &str, - leaf: Leaf, - value: FfiValue, - misfit: Error, -) -> Result<(), Error> { - let _ = match (leaf, value) { - (Leaf::U32, FfiValue::UInt32(v)) => values.insert(name, v), - (Leaf::Text, FfiValue::String(s)) => { - // Valid UTF-8 by `Utf8String`'s invariant; the bytes move, they - // are not copied. - let text = String::from_utf8(s.into_inner().risky_unwrap()).map_err(|_| misfit)?; - values.insert(name, text) - } - (Leaf::Value, value) => values.insert(name, Value::new(value)), - (Leaf::U32 | Leaf::Text, _) => return Err(misfit), - }; - Ok(()) -} - // ============================================================================= // Output adapters: the engine's record as the stored shape, and back // ============================================================================= @@ -1107,7 +1065,6 @@ fn insert_leaf( struct FieldShape { name: String, verb: Verb, - leaf: Leaf, keys: Vec<&'static str>, kind: Option, } @@ -1141,7 +1098,7 @@ fn shape_record( } Verb::Index => keyed_terms(values.take(&field.name)?, &field.keys)?, Verb::Passthrough => { - let value = take_leaf(&mut values, &field.name, field.leaf)?; + let value = take_leaf(&mut values, &field.name)?; vec![( "passthrough".to_string(), CipherText::Passthrough(Box::new(value) as BoxedPassthrough), @@ -1170,14 +1127,9 @@ fn keyed_terms( .collect()) } -/// The field `name` out of the record, as a value: the leaf type back to -/// the variant it came from. -fn take_leaf(values: &mut FieldValues, name: &str, leaf: Leaf) -> Result { - Ok(match leaf { - Leaf::U32 => FfiValue::UInt32(values.take(name)?), - Leaf::Text => FfiValue::String(values.take::(name)?.into()), - Leaf::Value => values.take::(name)?.into_inner(), - }) +/// The field `name` out of the record, as the value it holds. +fn take_leaf(values: &mut FieldValues, name: &str) -> Result { + Ok(values.take::(name)?.into_inner()) } /// The record the plan opened, as a value: the fields that come back, in @@ -1187,7 +1139,7 @@ fn take_leaf(values: &mut FieldValues, name: &str, leaf: Leaf) -> Result Result { let mut fields = Vec::with_capacity(shape.len()); for field in shape.iter().filter(|field| field.verb != Verb::Index) { - let value = take_leaf(&mut values, &field.name, field.leaf)?; + let value = take_leaf(&mut values, &field.name)?; if let Some(kind) = field.kind { if !kind.holds(&value) { return Err(crate::PlanError::FieldType { @@ -1246,7 +1198,7 @@ fn record_row(mut row: Vec<(String, StackCipherText)>, plan: &Plan) -> Result { let (_, ciphertext) = take(&mut outputs, "c").ok_or(Error::Record)?; @@ -1269,13 +1221,13 @@ mod tests { IdentifiedBy, IndexKey, IndexKeySource, RetrieveKeyPayload, UnverifiedContext, }; use uuid::Uuid; - use vitaminc_protected::Protected; + use vitaminc_protected::{Controlled, Protected}; use crate::dynamic::{context, term}; use crate::plan::pick; - use crate::sem::{EqualityTerm, MatchTerms, OreTerm}; + use crate::sem::EqualityTerm; use crate::target::{AeadContext, DecryptInto, EncryptFrom}; - use crate::{nonempty, Equality, Match, Ore, PlanError, StackCipher}; + use crate::{nonempty, Equality, PlanError, StackCipher}; /// `FakeDataKeySource` with call counters, so the batching contract — /// one key request per invocation, none for a refused call — is @@ -2442,34 +2394,39 @@ mod tests { } /// The lowering is the plan builder: the record a data plan seals is - /// the record the typed chain seals under the same declaration, field - /// for field — the same terms, and ciphertexts each side opens. + /// the record the chain seals under the same declaration over the + /// same plaintext type, a [`Value`] per field — the same terms, and + /// ciphertexts each side opens. #[tokio::test] - async fn seals_what_the_typed_chain_seals_under_the_same_declaration() { + async fn seals_what_the_chain_seals_under_the_same_declaration() { struct User { - age: u32, - email: String, - nick: String, + age: Value, + email: Value, + nick: Value, id: u64, } let cipher = cipher().await; let keyset = cipher.default_keyset(); let user = User { - age: 34, - email: "a@x".into(), - nick: "al smith".into(), + age: Value::new(FfiValue::UInt32(34)), + email: Value::new(s("a@x")), + nick: Value::new(s("al smith")), id: 7, }; - let mut typed = cipher + let default_match = IndexSpec::Match(crate::sem::MatchOptions::default()); + let mut chain = cipher .encrypt(&user) .context("users") .fields() - .encrypt_index(pick("age", |u: &User| &u.age), (Equality, Ore)) + .encrypt_index( + pick("age", |u: &User| &u.age), + (IndexSpec::Equality, IndexSpec::Ore), + ) .encrypt(pick("email", |u: &User| &u.email)) - .index(pick("nick", |u: &User| &u.nick), Match::default()) + .index(pick("nick", |u: &User| &u.nick), default_match) .passthrough(pick("id", |u: &User| &u.id)) .await - .expect("the typed chain"); + .expect("the chain"); let plan = plan(obj(vec![ ( "age", @@ -2482,45 +2439,36 @@ mod tests { .expect("plan"); let mut fields = map(seal(&keyset, row(34), &plan).await); - let age: Encrypted<(EqualityTerm, OreTerm)> = typed.take("age").expect("age"); + let age: Encrypted<(TermBytes, TermBytes)> = chain.take("age").expect("age"); let mut lowered_age = map(node(&mut fields, "age")); assert_eq!( term_bytes(&node(&mut lowered_age, "eq")), - age.terms.0.to_bytes(), + age.terms.0.as_bytes(), "the same equality term" ); assert_eq!( term_bytes(&node(&mut lowered_age, "ore")), - age.terms.1.to_bytes(), + age.terms.1.as_bytes(), "the same ore term" ); - let nick: MatchTerms = typed.take("nick").expect("nick"); + let nick: TermBytes = chain.take("nick").expect("nick"); let mut lowered_nick = map(node(&mut fields, "nick")); assert_eq!( term_bytes(&node(&mut lowered_nick, "match")), - nick.to_bytes(), + nick.as_bytes(), "the same match terms" ); // Each side's ciphertext opens through the other. - let typed_opened: u32 = cipher + let chain_opened: FfiValue = cipher .decrypt( node(&mut lowered_age, "c"), Label::parse("users/age").expect("label"), ) .await - .expect("the typed reader opens the lowering's u32"); - assert_eq!(typed_opened, 34); - let email: StackCipherText = typed.take("email").expect("email"); - let mut lowered_email = map(node(&mut fields, "email")); - let typed_email: String = cipher - .decrypt( - node(&mut lowered_email, "c"), - Label::parse("users/email").expect("label"), - ) - .await - .expect("the typed reader opens the lowering's string"); - assert_eq!(typed_email, "a@x"); + .expect("the chain's reader opens the lowering's leaf"); + assert_eq!(u32_of(&chain_opened), 34); + let email: StackCipherText = chain.take("email").expect("email"); let stored = CipherText::Map(vec![ ( "age".to_string(), @@ -2543,7 +2491,7 @@ mod tests { assert_eq!( u32_of(&opened[0].1), 34, - "the lowering opens the typed chain's u32" + "the lowering opens the chain's leaf" ); assert_eq!(text_of(&opened[1].1), "a@x", "and its string"); assert_eq!(u64_of(&opened[2].1), 7); @@ -2622,11 +2570,11 @@ mod tests { flat.to_bytes(), "the extension domain-separates from the flat label" ); - let opened: u32 = cipher + let opened: FfiValue = cipher .decrypt(node(&mut age, "c"), native) .await .expect("the leaf opens under the extended context"); - assert_eq!(opened, 34); + assert_eq!(u32_of(&opened), 34); } /// Several parts nest to the left, one at a time, as a binding @@ -3444,10 +3392,12 @@ mod tests { } } - /// The leaf encoding decision, pinned (ADR-0007): a field typed as a - /// kind with a Rust leaf type seals as that type seals, so a data plan - /// and a derive interchange; a field with no type seals the tagged - /// `FfiValue` encoding, which only a dynamic reader opens. + /// The leaf encoding, pinned: every field lowered from data seals the + /// tagged `Value` leaf whatever its `"type"`, so a type declared later + /// changes no bytes; a Rust field of a bare type shares a data field's + /// terms and not its leaf, and the two leaves cannot be told apart by + /// inspection, which is why the lowering never picks an encoding from + /// the type. mod given_the_leaf_encoding { use super::*; @@ -3458,11 +3408,106 @@ mod tests { hm: EqualityTerm, } + /// A `uint32` field and a `string` field seal the tagged leaf, as an + /// untyped field does: the leaf opens as a value, and a bare `u32` + /// reader refuses the five bytes it finds. + #[tokio::test] + async fn every_field_seals_the_tagged_leaf_whatever_its_type() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let age_label = Label::parse("users/age").expect("label"); + for ty in [None, Some("uint32")] { + let field = match ty { + Some(ty) => typed(label("age"), &["c"], ty), + None => spec(label("age"), &["c"]), + }; + let plan = plan(obj(vec![("age", field)])).expect("plan"); + let mut fields = + map(seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &plan).await); + let mut age = map(node(&mut fields, "age")); + let as_value: FfiValue = cipher + .decrypt(node(&mut age, "c"), age_label.clone()) + .await + .expect("the tagged leaf opens as a value"); + assert_eq!(u32_of(&as_value), 34, "type {ty:?}"); + let mut fields = + map(seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &plan).await); + let mut age = map(node(&mut fields, "age")); + let as_u32: Result = + cipher.decrypt(node(&mut age, "c"), age_label.clone()).await; + assert!( + matches!(as_u32, Err(crate::Error::Aead)), + "five tagged bytes are not a bare u32, type {ty:?}: {as_u32:?}" + ); + } + let plan = plan(obj(vec![( + "email", + typed(label("email"), &["c"], "string"), + )])) + .expect("plan"); + let mut fields = map(seal(&keyset, obj(vec![("email", s("a@x"))]), &plan).await); + let mut email = map(node(&mut fields, "email")); + let as_value: FfiValue = cipher + .decrypt( + node(&mut email, "c"), + Label::parse("users/email").expect("label"), + ) + .await + .expect("a tagged string leaf"); + assert_eq!(text_of(&as_value), "a@x"); + } + + /// Declaring a type on a field written without one, or dropping it, + /// changes no bytes: the row opens to the value it held, exactly, in + /// both directions. #1082 needs no re-encryption. + #[tokio::test] + async fn declaring_a_type_later_changes_no_bytes() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let untyped = plan(obj(vec![ + ("age", spec(label("age"), &["c"])), + ("email", spec(label("email"), &["c"])), + ])) + .expect("plan"); + let typed_plan = plan(obj(vec![ + ("age", typed(label("age"), &["c"], "uint32")), + ("email", typed(label("email"), &["c"], "string")), + ])) + .expect("plan"); + let source = || obj(vec![("age", FfiValue::UInt32(34)), ("email", s("alice"))]); + + let sealed_untyped = seal(&keyset, source(), &untyped).await; + let opened = object(open(&cipher, sealed_untyped, &typed_plan).await); + assert_eq!( + u32_of(&opened[0].1), + 34, + "an untyped row opens under a uint32 field" + ); + assert_eq!( + text_of(&opened[1].1), + "alice", + "an untyped row opens under a string field, with no leading byte" + ); + + let sealed_typed = seal(&keyset, source(), &typed_plan).await; + let opened = object(open(&cipher, sealed_typed, &untyped).await); + assert_eq!( + u32_of(&opened[0].1), + 34, + "a typed row opens under an untyped field" + ); + assert_eq!(text_of(&opened[1].1), "alice"); + } + /// A derived `plaintext = u32` record and a `uint32` data-plan field - /// under the same label: the same term, and each opens the other's - /// ciphertext. + /// under one label derive the same term. Their leaves are different + /// encodings, and neither reader opens the other's: the derive's bare + /// four bytes carry no tag for the lowering, the lowering's five + /// tagged bytes are not a `u32`. A Rust record that must interchange + /// leaves with a data plan declares `Value` fields instead + /// (`seals_what_the_chain_seals_under_the_same_declaration`). #[tokio::test] - async fn a_typed_field_and_the_derive_interchange() { + async fn the_derive_and_a_data_plan_share_terms_not_leaves() { let cipher = cipher().await; let keyset = cipher.default_keyset(); let label_ = Label::parse("users/age").expect("label"); @@ -3485,96 +3530,50 @@ mod tests { "the same equality term" ); - // The derive opens the lowering's leaf as a bare u32. - let opened: u32 = keyset + let as_u32: Result = keyset .decrypt_as( node(&mut age, "c"), AeadContext::from(NonEmpty::from(label_.clone())), ) - .await - .expect("a bare u32 leaf"); - assert_eq!(opened, 34); - - // The lowering opens the derive's leaf as a uint32 field. + .await; + assert!( + matches!(as_u32, Err(crate::Error::Aead)), + "the derive does not open the lowering's leaf: {as_u32:?}" + ); let stored = CipherText::Map(vec![( "age".to_string(), CipherText::Map(vec![("c".to_string(), derived.c)]), )]); - let opened = object(open(&cipher, stored, &plan).await); - assert_eq!(u32_of(&opened[0].1), 34); - } - - /// A field with no type seals the self-describing tagged encoding: - /// it reads back as what it was, and a bare `u32` reader does not - /// open it. The two encodings are different leaves, by declaration. - #[tokio::test] - async fn an_untyped_field_seals_the_tagged_encoding() { - let cipher = cipher().await; - let keyset = cipher.default_keyset(); - let label_ = Label::parse("users/age").expect("label"); - let untyped = plan(obj(vec![("age", spec(label("age"), &["c"]))])).expect("plan"); - let mut fields = - map(seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped).await); - let mut age = map(node(&mut fields, "age")); - let as_value: FfiValue = cipher - .decrypt(node(&mut age, "c"), label_.clone()) - .await - .expect("the tagged leaf opens as a value"); - assert_eq!(u32_of(&as_value), 34); - let mut fields = - map(seal(&keyset, obj(vec![("age", FfiValue::UInt32(34))]), &untyped).await); - let mut age = map(node(&mut fields, "age")); - let as_u32: Result = cipher.decrypt(node(&mut age, "c"), label_).await; - assert!( - as_u32.is_err(), - "five tagged bytes are not a bare u32: {as_u32:?}" - ); - - // And the typed leaf is not the tagged one. - let typed_plan = - plan(obj(vec![("age", typed(label("age"), &["c"], "uint32"))])).expect("plan"); - let mut fields = map(seal( - &keyset, - obj(vec![("age", FfiValue::UInt32(34))]), - &typed_plan, - ) - .await); - let mut age = map(node(&mut fields, "age")); - let as_value: Result = cipher - .decrypt( - node(&mut age, "c"), - Label::parse("users/age").expect("label"), - ) + let result = decrypt(Scope::Client(&cipher), stored, &plan) + .expect("the shape fits") .await; assert!( - as_value.is_err(), - "four bare bytes carry no tag for a value reader" + matches!(result, Err(crate::Error::Aead)), + "the lowering does not open the derive's leaf: {:?}", + result.err() ); } - /// A `string` field is a `String` leaf, as the derive's. + /// The two encodings cannot be told apart by inspection: a bare + /// `String` reader accepts a tagged string leaf, reading the tag as + /// a line feed. Pinned so that it is known, never relied on: the + /// lowering reads only its own encoding, and a Rust record that + /// shares rows with a binding declares `Value` fields. #[tokio::test] - async fn a_string_field_is_a_string_leaf() { + async fn a_bare_string_reader_cannot_tell_a_tagged_leaf_apart() { let cipher = cipher().await; let keyset = cipher.default_keyset(); - let plan = plan(obj(vec![( - "email", - typed(label("email"), &["c", "eq"], "string"), - )])) - .expect("plan"); - let mut fields = map(seal(&keyset, obj(vec![("email", s("a@x"))]), &plan).await); + let plan = plan(obj(vec![("email", spec(label("email"), &["c"]))])).expect("plan"); + let mut fields = map(seal(&keyset, obj(vec![("email", s("alice"))]), &plan).await); let mut email = map(node(&mut fields, "email")); - let label_ = Label::parse("users/email").expect("label"); - let opened: String = cipher - .decrypt(node(&mut email, "c"), label_.clone()) - .await - .expect("a bare string leaf"); - assert_eq!(opened, "a@x"); - let typed = keyset - .equality_term("a@x".to_string(), NonEmpty::from(label_)) + let bare: String = cipher + .decrypt( + node(&mut email, "c"), + Label::parse("users/email").expect("label"), + ) .await - .expect("typed"); - assert_eq!(term_bytes(&node(&mut email, "eq")), typed.to_bytes()); + .expect("valid UTF-8 either way"); + assert_eq!(bare, "\nalice", "the string tag, U+000A, read as text"); } } } diff --git a/packages/stack-encrypt/src/dynamic/term.rs b/packages/stack-encrypt/src/dynamic/term.rs index cf3d17017..5b9b46403 100644 --- a/packages/stack-encrypt/src/dynamic/term.rs +++ b/packages/stack-encrypt/src/dynamic/term.rs @@ -21,10 +21,7 @@ use zeroize::Zeroizing; use super::{utf8, Error, Value}; use crate::sem::{CllwOpeEncrypt, CllwOreEncrypt, DefaultMatch, MatchOptions, Tokenizer}; -use crate::target::{ - chosen, equality, ope as ope_op, ore as ore_op, CallerContext, ConsumeSource, Encryption, - Index, IndexSpec, Pending, -}; +use crate::target::{chosen, CallerContext, ConsumeSource, Encryption, Index, IndexSpec, Pending}; use crate::{IntoPrfContext, KeysetCipher, NonEmpty}; /// The runtime half of [`IndexSpec`]: the domain table, and the index's wire @@ -306,11 +303,11 @@ where /// positions, the raw CLLW bytes of an ORE or OPE term (see /// [`sem`](crate::sem)'s byte encodings). /// -/// The term type of every [`Index`] an [`IndexSpec`] implements: a term -/// derived through the dynamic path has no Rust term type to be, since the -/// index was named as data, so it is its bytes. Those bytes are exactly the -/// typed term's (`EqualityTerm::into_bytes`, `OreTerm::to_bytes`, …), -/// which is what makes a Rust-written term and a binding's probe compare. +/// The term type of [`IndexSpec`]'s [`Index`] impl: a term derived through +/// the dynamic path has no Rust term type to be, since the index was named +/// as data, so it is its bytes. Those bytes are exactly the typed term's +/// (`EqualityTerm::into_bytes`, `OreTerm::to_bytes`, …), which is what +/// makes a Rust-written term and a binding's probe compare. #[derive(Clone, PartialEq, Eq, Debug)] pub struct TermBytes(Vec); @@ -508,7 +505,9 @@ fn lifted(error: Error) -> crate::Error { /// `operation` is `scalar_term`'s dispatch — the one step of the dynamic /// path that stays dynamic — wrapped as a description, so a field lowered /// from data runs through the same `indexed()` and `zip` every other field -/// does, and its term is a [`TermBytes`]. +/// does, and its term is a [`TermBytes`]. A Rust chain over `Value` fields +/// names its indexes this way too (`(IndexSpec::Equality, IndexSpec::Ore)`), +/// and is then the same declaration as a data plan's. /// /// A value the scheme defines no such term for (a container, a float under /// equality) fails the description when it runs; a plan lowered from data @@ -539,62 +538,6 @@ impl Index for IndexSpec { } } -/// An [`IndexSpec`] is an [`Index`] of a `u32`: the typed index of the same -/// name — [`equality`], [`ore`](crate::target::ore), [`ope`](crate::target::ope) — -/// with its term as [`TermBytes`]. This is how a plan lowered from data runs -/// a field it knows to be a `u32` through exactly the operations the typed -/// chain's `encrypt_index::` runs. Match is not defined over an -/// integer, and fails the description when it runs; a plan refuses it when -/// it is built ([`admits`](super::admits)). -impl Index for IndexSpec { - type Term = TermBytes; - fn spec(&self) -> IndexSpec { - self.clone() - } - fn operation<'s, K: 'static, M: ConsumeSource<'s, u32>>( - &self, - ) -> Encryption<'s, u32, TermBytes, K, CallerContext, M> { - match self { - IndexSpec::Equality => equality::().map(equality_bytes), - IndexSpec::Ore => ore_op::().map(|term| TermBytes(term.to_bytes())), - IndexSpec::Ope => ope_op::().map(|term| TermBytes(term.to_bytes())), - IndexSpec::Match(_) => Encryption::failed(lifted(Error::Term { kind: self.clone() })), - } - } -} - -/// An [`IndexSpec`] is an [`Index`] of a `String`: the typed index of the -/// same name, with its term as [`TermBytes`]; see the `u32` impl. A match -/// index derives under the options the spec carries, which under the -/// defaults are `Match::default()`'s bytes. -impl Index for IndexSpec { - type Term = TermBytes; - fn spec(&self) -> IndexSpec { - self.clone() - } - fn operation<'s, K: 'static, M: ConsumeSource<'s, String>>( - &self, - ) -> Encryption<'s, String, TermBytes, K, CallerContext, M> { - match self { - IndexSpec::Equality => equality::().map(equality_bytes), - IndexSpec::Ore => ore_op::().map(|term| TermBytes(term.to_bytes())), - IndexSpec::Ope => ope_op::().map(|term| TermBytes(term.to_bytes())), - IndexSpec::Match(options) => { - let options = options.clone(); - chosen(move |source: M::Source, cipher, cx: CallerContext| { - let context = match cx.validated() { - Ok(context) => context, - Err(error) => return Pending::failed(cipher, error), - }; - cipher - .match_terms_under::(M::view(&source), context, options) - .map(|terms| TermBytes(terms.to_bytes())) - }) - } - } - } -} - #[cfg(test)] mod tests { use super::*; @@ -613,7 +556,7 @@ mod tests { /// A `TermBytes` reads as the typed term's bytes, through every /// accessor: `as_bytes`, `as_ref` and `into_bytes` give the same /// 32 PRF bytes an `EqualityTerm` holds, for a term derived through - /// `IndexSpec`'s `Index` and through the dynamic dispatch alike. + /// `IndexSpec`'s `Index`. #[tokio::test] async fn term_bytes_read_as_the_typed_terms_bytes() { let cipher = cipher().await; @@ -626,20 +569,6 @@ mod tests { .to_vec(); assert_eq!(typed.len(), 32); - let through_u32: TermBytes = keyset - .run( - Index::::operation::( - &IndexSpec::Equality, - ), - 34u32, - CallerContext::from(nonempty!("users/age")), - ) - .await - .expect("the u32 index derives"); - assert_eq!(through_u32.as_bytes(), typed.as_slice()); - assert_eq!(through_u32.as_ref(), typed.as_slice()); - assert_eq!(through_u32.clone().into_bytes(), typed); - let through_value: TermBytes = keyset .run( Index::::operation::( @@ -650,8 +579,9 @@ mod tests { ) .await .expect("the dynamic index derives"); - assert_eq!(through_value, through_u32, "one dispatch table, one term"); assert_eq!(through_value.as_bytes(), typed.as_slice()); + assert_eq!(through_value.as_ref(), typed.as_slice()); + assert_eq!(through_value.into_bytes(), typed); } /// A match index under the default options: what a plan's bare diff --git a/packages/stack-encrypt/src/dynamic/value.rs b/packages/stack-encrypt/src/dynamic/value.rs index 3e7079564..cd789f4cb 100644 --- a/packages/stack-encrypt/src/dynamic/value.rs +++ b/packages/stack-encrypt/src/dynamic/value.rs @@ -6,9 +6,9 @@ use vitaminc_aead::{Cipher, Decipher, Decrypt, Encrypt, IntoAad}; use vitaminc_aead_value::FfiValue; use vitaminc_protected::{Controlled, Protected}; -/// A runtime value as the plaintext of a plan field: what a field lowered -/// from data holds when its declared type names no Rust leaf type, or when -/// it declares none. +/// A runtime value as the plaintext of a plan field: what every field +/// lowered from data holds, whatever its declared type, and what a Rust +/// chain's field holds when its rows must open from a binding. /// /// The engine hands a borrowed field to every operation that consumes it /// and clones it on the way in, as it does a `String` field of a derived @@ -21,9 +21,11 @@ use vitaminc_protected::{Controlled, Protected}; /// each once per operation that consumes it, as the derive's does. /// /// It seals and opens as the [`FfiValue`] it wraps, in vitaminc's -/// self-describing tagged leaf encoding; [`dynamic::record`](super::record) -/// says when a field is this type and when it is a bare Rust leaf instead. -/// Its `Debug` names the type and nothing else: the value is plaintext. +/// self-describing tagged leaf encoding — a different leaf from a bare +/// `u32`'s or `String`'s, and one the other reader cannot tell apart by +/// inspection; [`dynamic::record`](super::record) says what follows from +/// that. Its `Debug` names the type and nothing else: the value is +/// plaintext. pub struct Value(FfiValue); impl Value { diff --git a/packages/stack-encrypt/src/target/index.rs b/packages/stack-encrypt/src/target/index.rs index bbadf6ae2..145cf0be7 100644 --- a/packages/stack-encrypt/src/target/index.rs +++ b/packages/stack-encrypt/src/target/index.rs @@ -375,7 +375,7 @@ macro_rules! single_index { } )+}; } -single_index!([] Equality, [O] Match, [] Ore, [] Ope); +single_index!([] Equality, [O] Match, [] Ore, [] Ope, [] IndexSpec); /// A tuple of indexes: each index's operation, zipped left to right, and the /// nested pairs `zip` builds flattened back into one tuple of terms. diff --git a/packages/stack-encrypt/tests/fixtures/README.md b/packages/stack-encrypt/tests/fixtures/README.md index 66f20a6e8..4f1390c35 100644 --- a/packages/stack-encrypt/tests/fixtures/README.md +++ b/packages/stack-encrypt/tests/fixtures/README.md @@ -1,16 +1,20 @@ # Record fixture: `record_lowering.json` -One declaration, two authors, one engine. The typed Rust chain -(`cipher.encrypt(&user).context("users").fields()…`) and the data-plan -lowering (`stack_encrypt::dynamic::record`, what a language binding -calls) each sealed the same plaintext under the same declaration. The -fixture holds both records, with their term bytes, sealed under a -deterministic key source so they open in any process built from the same -seed. `tests/record_lowering.rs` opens each record with the other author -and checks that both derive the terms it holds. That cross-opening is the -proof that `dynamic::record` is a lowering into the plan builder and not a -second executor (ADR-0007, amended 2026-10-06). A Go test reads the same -file later, once the Go SDK's generated code is the third author. +One declaration, two authors, one engine. A Rust chain +(`cipher.encrypt(&user).context("users").fields()…`, over fields of type +`stack_encrypt::dynamic::Value`, the plaintext type a binding's values have) +and the data-plan lowering (`stack_encrypt::dynamic::record`, what a +language binding calls) each sealed the same plaintext under the same +declaration. The fixture holds both records, with their term bytes, sealed +under a deterministic key source so they open in any process built from the +same seed. `tests/record_lowering.rs` opens each record with the other +author and checks that both derive the terms it holds. That cross-opening +is the proof that `dynamic::record` is a lowering into the plan builder and +not a second executor (ADR-0007, amended 2026-10-06). A Go test reads the +same file later, once the Go SDK's generated code is the third author. A +chain over bare `u32` or `String` fields derives the same terms but a +different leaf encoding, which the other reader cannot open or tell apart; +`dynamic::record`'s docs say why the lowering does not bridge that. Regenerate with `STACK_ENCRYPT_UPDATE_FIXTURES=1 cargo test -p stack-encrypt --all-features --test record_lowering`. Only the sealed bytes @@ -47,8 +51,9 @@ the file by hand. - **`plan`.** The declaration in the data grammar `dynamic::record::plan` parses: per field its label (a list of plain segments), its outputs (`"c"`, `"passthrough"`, or an index key `"eq"`, `"match"`, `"ore"`, `"ope"`) and its - `"type"` (a vitaminc `ValueKind` name). The typed chain writes the same - declaration as `Plan::context("users").fields()` with one verb per field. + `"type"` (a vitaminc `ValueKind` name). The Rust chain writes the same + declaration as `Plan::context("users").fields()` with one verb per field, + its indexes named as `IndexSpec`s over `Value` fields. - **`plaintext`.** The value both authors sealed, each field as JSON at the kind `plan` declares for it. - **`records`.** Each record in the stored shape, field by field, output by diff --git a/packages/stack-encrypt/tests/fixtures/record_lowering.json b/packages/stack-encrypt/tests/fixtures/record_lowering.json index 331f04908..e65af8cd2 100644 --- a/packages/stack-encrypt/tests/fixtures/record_lowering.json +++ b/packages/stack-encrypt/tests/fixtures/record_lowering.json @@ -60,12 +60,12 @@ "records": { "lowering": { "age": { - "c": "01000000000000000000000000000000004d5626148f77c4c6d9da0ddbdf7663a2200021162641133485216f389886cbd7c6e9b25afa0f5f166f985b23384e81ffcfac01e3f8f61316f918d46ed5980e418c10738cec2c74de854df7cf1d0c4cfdca1ce5", + "c": "01000000000000000000000000000000004d5626148f77c4c6d9da0ddbdf7663a2200021162641133485216f389886cbd7c6e9b25afa0f5f166f985b23384e81ffcfac010686f7787f57b4c7a2192c3499ea1b890f9f4623d84b72b5f40266b10b7f607a41", "eq": "5433335e609685a33b29c98369e68cd5d92e838aa566a3ff1df5c2d251b63885", "ore": "7ca54e16d7944c0a9d8577d87f3864fdf79e99fea5956ba956a126a605cbf3f1" }, "email": { - "c": "0100000000000000000000000000000000b2afa77357a61488146c0edaa0246f65200043956c4f4bba857028c28165dfce9c63d904b738bbb039b6e11c805112d4a1ee0121e959ee736d9143b3f96f6e9d98720cfdc9f143e9f2b2cac7b85dc791eab4f514d5053b85cfb2ec61836e", + "c": "0100000000000000000000000000000000b2afa77357a61488146c0edaa0246f65200043956c4f4bba857028c28165dfce9c63d904b738bbb039b6e11c805112d4a1ee01ec122b5ee8771a6a37fc17312ff3f7a56e93885e19d4bb189950aa37475eb74f63f5758c4f87a3494c5e46ff", "eq": "ec534cfbf2336785378c3761ff88623c46361b2ba7c5fa83930ebacae49cf646", "match": "06000d0013001d002200260029002a003000320043005a006000660068006c0077008f00960098009e00c200c500d600d900de00e300e700e900eb00ec00f200fa00" }, @@ -73,17 +73,17 @@ "passthrough": 42 }, "notes": { - "c": "0100000000000000000000000000000000723f7a7f14b07197168aa13bc52010b3200072a73194b45af604cc6201b5a57554b05cc7ed6ad22b9102cbcb71bd9067849601103f61ed1d241e16a6995f2af90fd47e5b61a68b3f9f750a1e902048648a746782626468d388" + "c": "0100000000000000000000000000000000723f7a7f14b07197168aa13bc52010b3200072a73194b45af604cc6201b5a57554b05cc7ed6ad22b9102cbcb71bd9067849601a2ae59c2cfca81b0ecde824090e2331f2c389202676855068652e9e566dfccb9a55e9973a9301c" } }, "typed_chain": { "age": { - "c": "010000000000000000000000000000000092e427c5e176b60261add6ce80fd02df20008d231453c97fdec97a0b6559ac8f509129d20163f262d40ce5d77cf1ebe0cad501c86e4a687dd91b4b0ac07922ff9b6195e2b1096d3b35bbc4ca54de28160dc164", + "c": "010000000000000000000000000000000092e427c5e176b60261add6ce80fd02df20008d231453c97fdec97a0b6559ac8f509129d20163f262d40ce5d77cf1ebe0cad501155707def46648a0c3685910bea6d3ec9c63aeb8c1e864092d47a3bd0269d135c9", "eq": "5433335e609685a33b29c98369e68cd5d92e838aa566a3ff1df5c2d251b63885", "ore": "7ca54e16d7944c0a9d8577d87f3864fdf79e99fea5956ba956a126a605cbf3f1" }, "email": { - "c": "0100000000000000000000000000000000cf92191950409f9fe82a2b11888ffc1d2000ca4d5bbafbff919af4d3a95c7cc046d632fd4c28785b238f4b6f7ea3aa08482901159eb270f977ce050a03335f53e1375462e3c42ab9d650374d6e1362ef3c89f7f065e4d1a80bd4111e0378", + "c": "0100000000000000000000000000000000cf92191950409f9fe82a2b11888ffc1d2000ca4d5bbafbff919af4d3a95c7cc046d632fd4c28785b238f4b6f7ea3aa08482901a3e671679e0e1ac106a28ad8a27e3ae143a6ad5c4b8280dbf6818d1bab32fa45cb0790df8551cec1c9bb3187", "eq": "ec534cfbf2336785378c3761ff88623c46361b2ba7c5fa83930ebacae49cf646", "match": "06000d0013001d002200260029002a003000320043005a006000660068006c0077008f00960098009e00c200c500d600d900de00e300e700e900eb00ec00f200fa00" }, @@ -91,7 +91,7 @@ "passthrough": 42 }, "notes": { - "c": "0100000000000000000000000000000000562a41e032316ff2d1264a9e9f5c3b842000de33c8d1f2caaab8b90f691161a57eeb1328f7f1c9b5be76dec83e8a53765f1101905680f4433c1de6c861814a0c40bb3cfc25a2f6fcbda1f82e75a4dae75f018e90aa16d294e9" + "c": "0100000000000000000000000000000000562a41e032316ff2d1264a9e9f5c3b842000de33c8d1f2caaab8b90f691161a57eeb1328f7f1c9b5be76dec83e8a53765f11015924202425d1e3be9754306c2e290e350232d5b5623190ebcc69feb55fb566fe4b4db024f1ebcc" } } } diff --git a/packages/stack-encrypt/tests/index.rs b/packages/stack-encrypt/tests/index.rs index 88939d992..bd9e0e5e9 100644 --- a/packages/stack-encrypt/tests/index.rs +++ b/packages/stack-encrypt/tests/index.rs @@ -38,17 +38,19 @@ fn caller() -> CallerContext { #[cfg(feature = "dynamic")] #[tokio::test] async fn a_vec_of_indexes_is_an_index_set_in_order_and_never_empty() { + use stack_encrypt::dynamic::{FfiValue, Value}; + let (cipher, generates, _) = counting_cipher().await; let keyset = cipher.default_keyset(); let list: Vec = vec![IndexSpec::Ore, IndexSpec::Equality]; assert_eq!( - Indexes::::specs(&list), + Indexes::::specs(&list), [IndexSpec::Ore, IndexSpec::Equality], "the specs are the indexes', in the order given" ); assert_eq!( - Indexes::::specs(&vec![IndexSpec::Equality]), + Indexes::::specs(&vec![IndexSpec::Equality]), [IndexSpec::Equality] ); assert_eq!( @@ -59,8 +61,8 @@ async fn a_vec_of_indexes_is_an_index_set_in_order_and_never_empty() { let terms: Vec = keyset .run( - Indexes::::operations::<_, Borrowed>(&list), - &34u32, + Indexes::::operations::<_, Borrowed>(&list), + &Value::new(FfiValue::UInt32(34)), caller(), ) .await diff --git a/packages/stack-encrypt/tests/record_lowering.rs b/packages/stack-encrypt/tests/record_lowering.rs index 3c3674809..09e374f2d 100644 --- a/packages/stack-encrypt/tests/record_lowering.rs +++ b/packages/stack-encrypt/tests/record_lowering.rs @@ -1,9 +1,13 @@ //! The record fixture is the proof of the lowering (ADR-0007, amended -//! 2026-10-06): the typed Rust chain and the data-plan lowering each open -//! the records the other sealed, and both derive the same bytes for each -//! term. `tests/fixtures/record_lowering.json` holds one record from each -//! author, sealed under a deterministic key source so the bytes open in any -//! process, and its README gives the schema a Go test reads later. +//! 2026-10-06): a Rust chain and the data-plan lowering each open the +//! records the other sealed, and both derive the same bytes for each term. +//! The chain's fields are `dynamic::Value`s, the plaintext type a binding's +//! values have, so the two authors write one declaration over one type; a +//! chain over bare `u32`/`String` fields shares the terms and not the leaves +//! (see `dynamic::record`). `tests/fixtures/record_lowering.json` holds one +//! record from each author, sealed under a deterministic key source so the +//! bytes open in any process, and its README gives the schema a Go test +//! reads later. //! //! Regenerate the fixture with `STACK_ENCRYPT_UPDATE_FIXTURES=1 cargo test //! --test record_lowering --all-features`; every other run reads it. @@ -17,41 +21,45 @@ use std::path::PathBuf; use common::{deterministic_cipher, DeterministicSource}; use serde_json::{json, Map, Value as Json}; use stack_encrypt::dynamic::record::{self, Plan as DataPlan}; -use stack_encrypt::dynamic::{FfiValue, Scope}; +use stack_encrypt::dynamic::{FfiValue, Scope, TermBytes, Value}; use stack_encrypt::plan::{pick, FieldValues}; -use stack_encrypt::sem::{EqualityTerm, MatchTerms, OreTerm}; -use stack_encrypt::target::Encrypted; -use stack_encrypt::{ - CipherText, Equality, Match, Ore, Plan, SealedValue, StackCipher, StackCipherText, -}; +use stack_encrypt::sem::MatchOptions; +use stack_encrypt::target::{Encrypted, IndexSpec}; +use stack_encrypt::{CipherText, Plan, SealedValue, StackCipher, StackCipherText}; use vitaminc_protected::Controlled; const SEED: [u8; 32] = *b"stack-encrypt record fixture v1 "; struct User { - age: u32, - email: String, - notes: String, + age: Value, + email: Value, + notes: Value, id: u32, } fn user() -> User { User { - age: 34, - email: "bob@example.com".into(), - notes: "likes cats".into(), + age: Value::new(FfiValue::UInt32(34)), + email: Value::new(FfiValue::String("bob@example.com".into())), + notes: Value::new(FfiValue::String("likes cats".into())), id: 42, } } -/// The one declaration, as the typed chain writes it. +/// The one declaration, as the Rust chain writes it over `Value` fields. fn typed_plan() -> Plan { Plan::context("users") .fields() - .encrypt_index(pick("age", |u: &User| &u.age), (Equality, Ore)) + .encrypt_index( + pick("age", |u: &User| &u.age), + (IndexSpec::Equality, IndexSpec::Ore), + ) .encrypt_index( pick("email", |u: &User| &u.email), - (Equality, Match::default()), + ( + IndexSpec::Equality, + IndexSpec::Match(MatchOptions::default()), + ), ) .encrypt(pick("notes", |u: &User| &u.notes)) .passthrough(pick("id", |u: &User| &u.id)) @@ -70,8 +78,7 @@ fn data_plan_json() -> Json { } fn plaintext_json() -> Json { - let u = user(); - json!({ "age": u.age, "email": u.email, "notes": u.notes, "id": u.id }) + json!({ "age": 34, "email": "bob@example.com", "notes": "likes cats", "id": 42 }) } // ---- JSON <-> FfiValue, for the subset the fixture uses -------------------- @@ -221,8 +228,8 @@ fn record_of_json(record: &Json) -> StackCipherText { /// The typed chain's record, shaped as the stored tree, by hand: what a Go /// program assembles from the engine's standard outputs. fn record_of_typed(mut values: FieldValues) -> StackCipherText { - let age: Encrypted<(EqualityTerm, OreTerm)> = values.take("age").expect("age"); - let email: Encrypted<(EqualityTerm, MatchTerms)> = values.take("email").expect("email"); + let age: Encrypted<(TermBytes, TermBytes)> = values.take("age").expect("age"); + let email: Encrypted<(TermBytes, TermBytes)> = values.take("email").expect("email"); let notes: StackCipherText = values.take("notes").expect("notes"); let id: u32 = values.take("id").expect("id"); let term = |bytes: Vec| -> StackCipherText { @@ -236,16 +243,16 @@ fn record_of_typed(mut values: FieldValues) -> StackCipherText { "age".into(), CipherText::Map(vec![ ("c".into(), age.ciphertext), - ("eq".into(), term(age.terms.0.to_bytes())), - ("ore".into(), term(age.terms.1.to_bytes())), + ("eq".into(), term(age.terms.0.into_bytes())), + ("ore".into(), term(age.terms.1.into_bytes())), ]), ), ( "email".into(), CipherText::Map(vec![ ("c".into(), email.ciphertext), - ("eq".into(), term(email.terms.0.to_bytes())), - ("match".into(), term(email.terms.1.to_bytes())), + ("eq".into(), term(email.terms.0.into_bytes())), + ("match".into(), term(email.terms.1.into_bytes())), ]), ), ("notes".into(), CipherText::Map(vec![("c".into(), notes)])), @@ -344,10 +351,13 @@ async fn open_typed(cipher: &StackCipher, tree: StackCipher .using(&typed_plan()) .await .expect("the typed chain opens it"); + let mut value = + |name: &str| json_of_opened(opened.take::(name).expect(name).into_inner()); + let (age, email, notes) = (value("age"), value("email"), value("notes")); json!({ - "age": opened.take::("age").expect("age"), - "email": opened.take::("email").expect("email"), - "notes": opened.take::("notes").expect("notes"), + "age": age, + "email": email, + "notes": notes, "id": opened.take::("id").expect("id"), }) } From cceda75174b9dc88ec4cb70b42cc060f87485423 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 02:17:42 -0700 Subject: [PATCH 08/14] fix(stack-encrypt): an engine slot the plan did not produce is ResponseShape MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit cipherstash-bot, on PR #1093: the output adapters read the record the engine built with FieldValues::take, whose refusals are PlanError::NotInValue and PlanError::FieldType; the ? turned those into Error::Plan, which the guest reports as STATUS_ENCODING, "your input is wrong". Only a bug in this module can make the engine's record disagree with the plan it was built from, so the host would have looked for a fault in its own data and found none. The adapters now take slots through one helper that reports a missing or mistyped slot as Error::ResponseShape — what a miscounted term list in the same file already is — which the guest maps to STATUS_INTERNAL. The only Error::Plan the adapters still raise is the kind check on an opened value, which is about the caller's data. A unit test drives both adapters with a missing and a mistyped slot. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- packages/stack-encrypt/src/dynamic/record.rs | 57 ++++++++++++++++++-- 1 file changed, 53 insertions(+), 4 deletions(-) diff --git a/packages/stack-encrypt/src/dynamic/record.rs b/packages/stack-encrypt/src/dynamic/record.rs index 02de226b7..8a704b4af 100644 --- a/packages/stack-encrypt/src/dynamic/record.rs +++ b/packages/stack-encrypt/src/dynamic/record.rs @@ -1069,6 +1069,15 @@ struct FieldShape { kind: Option, } +/// A slot of the record the engine built from the plan, at the type the +/// plan's verb produces. A missing or mistyped slot is the engine answering +/// with a shape other than the one it was asked for — a bug here, never the +/// caller's input — so it is [`ResponseShape`](crate::Error::ResponseShape), +/// as a miscounted term list is, and not a plan refusal. +fn slot(values: &mut FieldValues, name: &str) -> Result { + values.take(name).map_err(|_| crate::Error::ResponseShape) +} + /// A term as the stored tree carries it: a passthrough byte node. fn term_node(term: TermBytes) -> StackCipherText { CipherText::Passthrough(Box::new(FfiValue::Bytes(vitaminc_protected::Protected::new( @@ -1086,17 +1095,17 @@ fn shape_record( for field in shape { let outputs = match field.verb { Verb::Encrypt => { - let ciphertext: StackCipherText = values.take(&field.name)?; + let ciphertext: StackCipherText = slot(&mut values, &field.name)?; vec![("c".to_string(), ciphertext)] } Verb::EncryptIndex => { - let sealed: Encrypted> = values.take(&field.name)?; + let sealed: Encrypted> = slot(&mut values, &field.name)?; let mut outputs = Vec::with_capacity(1 + field.keys.len()); outputs.push(("c".to_string(), sealed.ciphertext)); outputs.extend(keyed_terms(sealed.terms, &field.keys)?); outputs } - Verb::Index => keyed_terms(values.take(&field.name)?, &field.keys)?, + Verb::Index => keyed_terms(slot(&mut values, &field.name)?, &field.keys)?, Verb::Passthrough => { let value = take_leaf(&mut values, &field.name)?; vec![( @@ -1129,7 +1138,7 @@ fn keyed_terms( /// The field `name` out of the record, as the value it holds. fn take_leaf(values: &mut FieldValues, name: &str) -> Result { - Ok(values.take::(name)?.into_inner()) + Ok(slot::(values, name)?.into_inner()) } /// The record the plan opened, as a value: the fields that come back, in @@ -1462,6 +1471,46 @@ mod tests { /// expectation is a predicate. type Refused = (&'static str, FfiValue, fn(&Error) -> bool); + /// The adapters read the record the engine built from the plan. A slot + /// that is missing or of another type than the plan's verb produces is + /// the engine's shape disagreeing with the plan's — this module's bug — + /// and is reported as `ResponseShape`, which a binding maps to its + /// internal status, never as a plan refusal the caller is told to fix. + #[test] + fn a_slot_the_engine_did_not_produce_is_a_response_shape_error() { + let shape = the_plan().shape(); + let empty = FieldValues::new(); + assert!( + matches!( + shape_record(empty, &shape), + Err(crate::Error::ResponseShape) + ), + "a missing slot on the encrypt side" + ); + let mut wrong = FieldValues::new(); + let _ = wrong.insert("age", 34u32); + assert!( + matches!( + shape_record(wrong, &shape), + Err(crate::Error::ResponseShape) + ), + "a mistyped slot on the encrypt side" + ); + let mut wrong = FieldValues::new(); + let _ = wrong.insert("age", 34u32); + assert!( + matches!(open_record(wrong, &shape), Err(crate::Error::ResponseShape)), + "a mistyped slot on the decrypt side" + ); + assert!( + matches!( + open_record(FieldValues::new(), &shape), + Err(crate::Error::ResponseShape) + ), + "a missing slot on the decrypt side" + ); + } + mod given_a_plan_value { use super::*; From d4e8d862b6f4d755df7de735cad77ff77f0a209a Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 02:18:15 -0700 Subject: [PATCH 09/14] test(stack-encrypt): every index arm of a typed field, and a refused pair cipherstash-bot, on PR #1093: no test ran the ORE and OPE arms over a typed uint32 or string field, nor a non-default match index over a typed string, and nothing ran the failure branch of IndexSpec's public Index impl. The typed-leaf arms are gone with the one-encoding fix, but the concern stands for the Index dispatch and the output key each term rides under: an arm calling the wrong operation still compiles, and the stored terms would then match no probe, with no error. A table test seals a typed uint32 and string field under each index the kind admits (equality, ORE, OPE, default and wide match) and checks the stored term is the one dynamic::term derives under the field's context. A second test runs IndexSpec as an Index directly over three pairs the scheme refuses and checks the run fails with the dynamic Error::Term, naming the index, inside Error::Other. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- packages/stack-encrypt/src/dynamic/record.rs | 51 ++++++++++++++++++++ packages/stack-encrypt/src/dynamic/term.rs | 46 ++++++++++++++++++ 2 files changed, 97 insertions(+) diff --git a/packages/stack-encrypt/src/dynamic/record.rs b/packages/stack-encrypt/src/dynamic/record.rs index 8a704b4af..02e4c3645 100644 --- a/packages/stack-encrypt/src/dynamic/record.rs +++ b/packages/stack-encrypt/src/dynamic/record.rs @@ -3395,6 +3395,57 @@ mod tests { ); } + /// Every index a typed field can declare stores the term a binding's + /// probe derives under the field's context: ORE and OPE over a + /// `uint32` and a `string`, and a match index under non-default + /// options over a `string`, through the output key each rides under. + #[tokio::test] + async fn a_typed_field_stores_the_term_the_dynamic_probe_derives() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let wide = IndexSpec::Match(crate::sem::MatchOptions { + k: 6, + m: 1024, + ..crate::sem::MatchOptions::default() + }); + for (ty, kind) in [ + ("uint32", IndexSpec::Equality), + ("uint32", IndexSpec::Ore), + ("uint32", IndexSpec::Ope), + ("string", IndexSpec::Equality), + ("string", IndexSpec::Ore), + ("string", IndexSpec::Ope), + ( + "string", + IndexSpec::Match(crate::sem::MatchOptions::default()), + ), + ("string", wide), + ] { + let value = || match ty { + "uint32" => FfiValue::UInt32(34), + _ => s("al smith"), + }; + let plan = plan(obj(vec![( + "f", + obj(vec![ + ("context", label("f")), + ("outputs", FfiValue::Array(vec![s("c"), kind.to_value()])), + ("type", s(ty)), + ]), + )])) + .expect("plan"); + let mut fields = map(seal(&keyset, obj(vec![("f", value())]), &plan).await); + let mut f = map(node(&mut fields, "f")); + let stored = term_bytes(&node(&mut f, kind.key())); + let scalar = Scalar::of(&value(), &kind).expect("scalar"); + let probe = term(&keyset, scalar, &kind, plan.fields()[0].context().clone()) + .await + .expect("probe"); + assert_eq!(stored, probe, "{ty} {kind}"); + assert!(!stored.is_empty(), "{ty} {kind}"); + } + } + /// A passthrough field with a declared type carries only values of /// that type, in and out. #[tokio::test] diff --git a/packages/stack-encrypt/src/dynamic/term.rs b/packages/stack-encrypt/src/dynamic/term.rs index 5b9b46403..6eccce49a 100644 --- a/packages/stack-encrypt/src/dynamic/term.rs +++ b/packages/stack-encrypt/src/dynamic/term.rs @@ -553,6 +553,52 @@ mod tests { .expect("build cipher") } + /// A pair the scheme refuses fails the run of `IndexSpec`'s public + /// `Index` with the dynamic `Error::Term` inside the crate error, + /// and requests nothing: the record path refuses such a pair at its + /// boundary, a Rust caller running the index directly meets it here. + #[tokio::test] + async fn a_refused_pair_fails_the_index_when_it_runs() { + let cipher = cipher().await; + let keyset = cipher.default_keyset(); + let ctx = || CallerContext::from(nonempty!("users/age")); + for (what, value, kind) in [ + ( + "a float under equality", + FfiValue::Float64(1.5), + IndexSpec::Equality, + ), + ( + "an integer under match", + FfiValue::UInt32(34), + default_match(), + ), + ( + "a container under ore", + FfiValue::Array(vec![FfiValue::UInt32(1)]), + IndexSpec::Ore, + ), + ] { + let refused = keyset + .run( + Index::::operation::(&kind), + Value::new(value), + ctx(), + ) + .await; + match refused { + Err(crate::Error::Other(inner)) => assert!( + matches!( + inner.downcast_ref::(), + Some(Error::Term { kind: refused_kind }) if *refused_kind == kind + ), + "{what}: the lifted error names the index: {inner:?}" + ), + other => panic!("{what}: expected the lifted Error::Term, got {other:?}"), + } + } + } + /// A `TermBytes` reads as the typed term's bytes, through every /// accessor: `as_bytes`, `as_ref` and `into_bytes` give the same /// 32 PRF bytes an `EqualityTerm` holds, for a term derived through From e92ca5a0471f845d73ae0941df0727fb5e285109 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 02:18:15 -0700 Subject: [PATCH 10/14] test(go): the two plan rules NewPlan leaves to the record call are refused there MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit cipherstash-bot, on PR #1093: NewPlan documents two rules it leaves to the call — every field under one table, no two fields under one label — and no Go test showed the call refusing them. One does now, on an uninitialised guest: both plans build, and EncryptRecords and DecryptRecord each refuse them as ErrEncoding before looking for a cipher. If the guest's refusal ever stops being the caller's input, this is the test that says so. Claude-Session: https://claude.ai/code/session_01V3WFXwax4J3uecpFEJ6yHc --- languages/golang/stackencrypt/guest_test.go | 40 +++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/languages/golang/stackencrypt/guest_test.go b/languages/golang/stackencrypt/guest_test.go index c71f59ffb..770738ea7 100644 --- a/languages/golang/stackencrypt/guest_test.go +++ b/languages/golang/stackencrypt/guest_test.go @@ -762,6 +762,46 @@ func TestGuestRefusesMalformedInputsBeforeState(t *testing.T) { } } +// NewPlan leaves two rules to the record call: every field under one table +// (the label's leading segments), and no two fields under one label. The +// guest's lowering refuses both at parse, before it looks for a cipher, as +// the caller's input (ErrEncoding), on the write and on the read. +func TestGuestRefusesAPlanNewPlanLeavesToTheCall(t *testing.T) { + ctx := context.Background() + c := rawInstance(t) + type row struct { + Age uint32 + Email string + } + record := EncryptedRecord{ + "Age": {Ciphertext: Sealed(fixtureLeaf)}, + "Email": {Ciphertext: Sealed(fixtureLeaf)}, + } + for name, fields := range map[string][]FieldPlan{ + "two tables": { + {Field: "Age", Context: label(t, "users/age").Context()}, + {Field: "Email", Context: label(t, "accounts/email").Context()}, + }, + "one label twice": { + {Field: "Age", Context: label(t, "users/age").Context(), Terms: []TermKind{Equality}}, + {Field: "Email", Context: label(t, "users/age").Context(), Terms: []TermKind{Equality}}, + }, + } { + p, err := NewPlan(fields...) + if err != nil { + t.Fatalf("%s: NewPlan refused it: %v", name, err) + } + _, err = c.DefaultKeyset().EncryptRecords(ctx, []row{{Age: 1, Email: "a@b.c"}}, WithPlan(p)) + if !errors.Is(err, ErrEncoding) { + t.Errorf("%s: EncryptRecords err = %v, want ErrEncoding", name, err) + } + var out row + if err := c.DecryptRecord(ctx, record, &out, WithPlan(p)); !errors.Is(err, ErrEncoding) { + t.Errorf("%s: DecryptRecord err = %v, want ErrEncoding", name, err) + } + } +} + // A bad ExtendContext part fails the call for that reason, on the probe and // on both record directions. TestGuestRefusesMalformedInputsBeforeState // shows the call never reaches the cipher, but a call that ignored the From fd6d618a66ca19668241bcd554302895888f6837 Mon Sep 17 00:00:00 2001 From: Lindsay Holmwood Date: Wed, 7 Oct 2026 14:40:44 +1100 Subject: [PATCH 11/14] test(stack-encrypt): the label-binding test swaps two fields only the key source can refuse cipherstash-bot, on PR #1093: the_fixture_records_are_bound_to_their_labels swapped age (uint32) and notes (string) and asserted only is_err(). Both fields declare a "type", so the per-field type check fails that decrypt whether or not the key source refuses the label, and the test could not tell the two apart. It now swaps email and notes, both string fields, so the type check passes and only the label binding can refuse, and it requires Error::Kms. With the deterministic source's descriptor check disabled the test fails with Error::Aead; is_err() would have passed. Claude-Session: https://claude.ai/code/session_016CfyFnsnw5NWKPojCQRxM9 --- .../stack-encrypt/tests/record_lowering.rs | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/packages/stack-encrypt/tests/record_lowering.rs b/packages/stack-encrypt/tests/record_lowering.rs index 09e374f2d..09ab5d4c9 100644 --- a/packages/stack-encrypt/tests/record_lowering.rs +++ b/packages/stack-encrypt/tests/record_lowering.rs @@ -491,20 +491,23 @@ async fn the_fixture_records_are_bound_to_their_labels() { let CipherText::Map(mut fields) = sealed else { panic!("a map"); }; - // Swap the `age` and `notes` field names: each ciphertext now sits under - // the other's label. + // `email` and `notes` are both `string` fields, so the per-field type + // check cannot refuse the swap: only the key source's label binding can. for (name, _) in &mut fields { *name = match name.as_str() { - "age" => "notes".into(), - "notes" => "age".into(), + "email" => "notes".into(), + "notes" => "email".into(), other => other.into(), }; } let result = record::decrypt(Scope::Client(&cipher), CipherText::Map(fields), &plan) .expect("the shape fits") .await; - assert!( - result.is_err(), - "a leaf under another field's label does not open" - ); + match result { + Err(stack_encrypt::Error::Kms(_)) => {} + Err(other) => { + panic!("a leaf under another field's label is refused by the key source, got {other:?}") + } + Ok(_) => panic!("a leaf under another field's label opened"), + } } From bb91a4eda68b13d03c1d2b3053fc54d99f637ab3 Mon Sep 17 00:00:00 2001 From: Lindsay Holmwood Date: Wed, 7 Oct 2026 14:42:25 +1100 Subject: [PATCH 12/14] docs(stack-encrypt): the leaf-encoding decision is tracked in #1118, not #1082 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The dynamic::record module docs and the CHANGELOG left the remaining part of #1059 item 4 — a Rust u32 or String field and a data-plan field seal different leaf bytes under one label — to #1082. #1082 is a different problem (an indexed field with no "type" storing two terms for one value), never mentions leaf encoding, and #1096 closes it, which would have left the decision pointing at a closed issue that never tracked it. #1118 now holds it: the two hazards (a bare String reader returns "\nalice"; a u32 mismatch fails as Error::Aead, which the Go guest reports as tampering) and the two ways to close them. Both docs point there. Claude-Session: https://claude.ai/code/session_016CfyFnsnw5NWKPojCQRxM9 --- packages/stack-encrypt/CHANGELOG.md | 2 +- packages/stack-encrypt/src/dynamic/record.rs | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/stack-encrypt/CHANGELOG.md b/packages/stack-encrypt/CHANGELOG.md index c5156ade0..8f0ecf0a6 100644 --- a/packages/stack-encrypt/CHANGELOG.md +++ b/packages/stack-encrypt/CHANGELOG.md @@ -61,7 +61,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 declares `dynamic::Value` fields, which are the same declaration as a data plan's. One leaf encoding for both authors, or the encoding bound into the leaf's context so the wrong reader fails closed, is a change to the - Rust chain's bytes and is left to #1082. + Rust chain's bytes, tracked in #1118. - `dynamic::Output` gains `Passthrough` (the wire key `"passthrough"`). The enum is exhaustive on purpose, so a match over it must name the variant. - `target::DeclaredContext` holds several extension parts: `with(part)` diff --git a/packages/stack-encrypt/src/dynamic/record.rs b/packages/stack-encrypt/src/dynamic/record.rs index 02e4c3645..ea7fcdfb6 100644 --- a/packages/stack-encrypt/src/dynamic/record.rs +++ b/packages/stack-encrypt/src/dynamic/record.rs @@ -62,7 +62,7 @@ //! terms and nothing else. Closing that gap — one leaf encoding both authors //! read, or the encoding bound into the leaf's context so the wrong reader //! fails closed — is a change to the Rust chain's bytes, and so a decision -//! recorded against #1082, not something the lowering can take on its own. +//! tracked in #1118, not something the lowering can take on its own. //! //! # The stored record is wire format //! From 990d8398612e1c7539ce36792c20503deb949839 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 21:04:03 -0700 Subject: [PATCH 13/14] fix(go): plantest spells a context as its segments, so a Custom change shows plan.Custom("notes/v1") now binds the label ["notes", "v1"] rather than the one text part "notes/v1". The snapshot wrote both as `notes/v1`, so the golden file did not change and a reviewer saw nothing. A context line is now the label's segments, `context ["individuals-notes", "v1"]`, and parse refuses the old joined spelling rather than guess which shape it meant. An EQL column and a Custom column with the same segments now bind the same context, so the kind is no longer part of contextKey: switching a column from EQL to Custom("
/") is a target change under the same context, not the data loss the snapshot used to report. The CHANGELOG entry that names the context= tag names the Custom change. --- languages/golang/stackencrypt/README.md | 4 +- .../stackencrypt/plan/plantest/compare.go | 45 +++---- .../plan/plantest/plantest_internal_test.go | 92 ++++++++------- .../stackencrypt/plan/plantest/snapshot.go | 111 +++++++++++++----- .../testdata/TestPolicies/individuals.golden | 8 +- packages/stack-encrypt/CHANGELOG.md | 9 +- 6 files changed, 162 insertions(+), 107 deletions(-) diff --git a/languages/golang/stackencrypt/README.md b/languages/golang/stackencrypt/README.md index 88c9765e1..122e86e21 100644 --- a/languages/golang/stackencrypt/README.md +++ b/languages/golang/stackencrypt/README.md @@ -360,13 +360,13 @@ This is the golden file for the `Individuals` policy above: table individuals column email - context individuals/email + context ["individuals", "email"] target EQL terms eq match fact fides.data_categories user.contact.email column medicare_number - context individuals/medicare_number + context ["individuals", "medicare_number"] target EQL terms eq fact fides.data_categories user.government_id diff --git a/languages/golang/stackencrypt/plan/plantest/compare.go b/languages/golang/stackencrypt/plan/plantest/compare.go index dda05b005..ecb892cad 100644 --- a/languages/golang/stackencrypt/plan/plantest/compare.go +++ b/languages/golang/stackencrypt/plan/plantest/compare.go @@ -4,7 +4,6 @@ import ( "fmt" "path/filepath" "slices" - "strconv" "strings" "github.com/cipherstash/stack/languages/golang/stackencrypt/plan" @@ -88,7 +87,7 @@ func compare(old, cur snapshot, facts []plan.Fact, m plan.Message) changes { seenCol[n.name] = true if !sameContext(o, n) { c.contexts = append(c.contexts, fmt.Sprintf("column %s: its context is %s, was %s. %s", - token(o.name), spellContext(n, o), spellContext(o, n), pinAdvice(m, facts, n.from, o, old.table))) + token(o.name), n.context, o.context, pinAdvice(m, facts, n.from, o, old.table))) } c.stored(o, n) continue @@ -99,7 +98,7 @@ func compare(old, cur snapshot, facts []plan.Fact, m plan.Message) changes { // between them. if n, ok := only(cur.columns, func(n column) bool { _, before := oldCols[n.name] - return !before && !seenCol[n.name] && n.context == o.context && o.kind == kindEQL && n.kind == kindEQL + return !before && !seenCol[n.name] && sameContext(n, o) && o.kind == kindEQL && n.kind == kindEQL }); ok { seenCol[n.name] = true c.migrations = append(c.migrations, fmt.Sprintf("column %s is now stored in column %s, under the same context: the database column is renamed with it (ALTER TABLE ... RENAME COLUMN).", @@ -114,7 +113,7 @@ func compare(old, cur snapshot, facts []plan.Fact, m plan.Message) changes { return !before && !seenPlain[p.field] && (p.field == o.name || len(o.facts) > 0 && slices.Equal(p.facts, o.facts)) }); ok { seenPlain[p.field] = true - c.migrations = append(c.migrations, fmt.Sprintf("column %s is now field %s, decided Plaintext. Rows already written hold ciphertexts under %q, which a migration must decrypt before the field is read as plaintext.", + c.migrations = append(c.migrations, fmt.Sprintf("column %s is now field %s, decided Plaintext. Rows already written hold ciphertexts under %s, which a migration must decrypt before the field is read as plaintext.", token(o.name), token(p.field), o.context)) continue } @@ -126,7 +125,7 @@ func compare(old, cur snapshot, facts []plan.Fact, m plan.Message) changes { if len(writers) > 1 { still = "columns " + strings.Join(writers, ", ") + " still write it" } - c.other = append(c.other, fmt.Sprintf("column %s is no longer written. Its context %q is not lost, since %s, but no field reads column %s now: if its field was renamed, pin the renamed field's rule with plan.Column(%q).", + c.other = append(c.other, fmt.Sprintf("column %s is no longer written. Its context %s is not lost, since %s, but no field reads column %s now: if its field was renamed, pin the renamed field's rule with plan.Column(%q).", token(o.name), o.context, still, token(o.name), o.name)) continue } @@ -136,17 +135,17 @@ func compare(old, cur snapshot, facts []plan.Fact, m plan.Message) changes { // carry the same ones, and pinning it would store two fields in one // column. So the guess states both readings, and the column is // still listed as new. - msg := fmt.Sprintf("column %s: no field writes its context %q any more.", token(o.name), o.context) + msg := fmt.Sprintf("column %s: no field writes its context %s any more.", token(o.name), o.context) if n, ok := only(cur.columns, func(n column) bool { _, before := oldCols[n.name] return !before && !seenCol[n.name] && !oldContexts[contextKey(n)] && len(o.facts) > 0 && slices.Equal(n.facts, o.facts) }); ok { seenCol[n.name] = true - msg += fmt.Sprintf(" Field %s now writes column %s under %q with the same facts, so it may be the same field renamed: %s"+ + msg += fmt.Sprintf(" Field %s now writes column %s under %s with the same facts, so it may be the same field renamed: %s"+ " If %s is instead a new field and %s was removed, do not pin it: that would store two fields in column %s under one context.", fieldName(n.from), token(n.name), n.context, pinAdvice(m, facts, n.from, o, old.table), token(n.name), token(o.name), token(o.name)) - c.other = append(c.other, fmt.Sprintf("new column %s, under %q, with terms [%s]. It is also named above as a possible rename of column %s.", + c.other = append(c.other, fmt.Sprintf("new column %s, under %s, with terms [%s]. It is also named above as a possible rename of column %s.", token(n.name), n.context, termList(n.terms), token(o.name))) c.stored(o, n) } else { @@ -163,13 +162,13 @@ func compare(old, cur snapshot, facts []plan.Fact, m plan.Message) changes { if p, ok := oldPlain[n.from.fact.Field]; ok { if _, still := curPlain[p.field]; !still { seenOldPlain[p.field] = true - c.migrations = append(c.migrations, fmt.Sprintf("field %s, decided Plaintext before, is now encrypted into column %s under %q. Rows already written hold its plaintext, which a migration must encrypt.", + c.migrations = append(c.migrations, fmt.Sprintf("field %s, decided Plaintext before, is now encrypted into column %s under %s. Rows already written hold its plaintext, which a migration must encrypt.", token(p.field), token(n.name), n.context)) continue } } } - c.other = append(c.other, fmt.Sprintf("new column %s, under %q, with terms [%s].", token(n.name), n.context, termList(n.terms))) + c.other = append(c.other, fmt.Sprintf("new column %s, under %s, with terms [%s].", token(n.name), n.context, termList(n.terms))) } for _, o := range old.plaintext { @@ -210,28 +209,15 @@ func (c *changes) stored(o, n column) { } } -// contextKey names a column's context exactly: its text and its target -// kind. An EQL context is the pair (table, column identity) and a Custom -// one is a single text part, so an EQL column and a Custom one never share -// a context, even when the Custom text reads "
/". -func contextKey(c column) string { return c.kind + " " + c.context } +// contextKey names a column's context exactly: its segments. An EQL +// context and a Custom one are both labels, so the two kinds share a +// context when their segments match; the kind is how the context was +// chosen, not part of it. +func contextKey(c column) string { return c.context } // sameContext reports whether a and b bind the same context. func sameContext(a, b column) bool { return contextKey(a) == contextKey(b) } -// spellContext quotes c's context for a message comparing it with other's. -// When the two read the same but differ in kind, it says which shape each -// is, since the text alone would read as no change. -func spellContext(c, other column) string { - if c.context != other.context || c.kind == other.kind { - return strconv.Quote(c.context) - } - if c.kind == kindEQL { - return fmt.Sprintf("%q (the EQL table/column pair)", c.context) - } - return fmt.Sprintf("%q (one Custom text part)", c.context) -} - // pinAdvice says how to store the field from in the old column under the // old context again, each answer checked by building the plan with it: the // old table, when the table moved, with a pin if it needs one; a pin; or @@ -276,7 +262,8 @@ func pin(m plan.Message, table plan.Table, facts []plan.Fact, from *decided, old tries = append(tries, try{}) } tries = append(tries, try{fmt.Sprintf("plan.Column(%q)", column), []plan.RuleOption{plan.Column(column)}}) - if id, ok := strings.CutPrefix(context, string(table)+"/"); ok && id != "" { + if segments, err := segmentsOf(context); err == nil && len(segments) == 2 && segments[0] == string(table) { + id := segments[1] tries = append(tries, try{fmt.Sprintf("plan.Identity(%q)", id), []plan.RuleOption{plan.Identity(id)}}, try{fmt.Sprintf("plan.Column(%q), plan.Identity(%q)", column, id), []plan.RuleOption{plan.Column(column), plan.Identity(id)}}, diff --git a/languages/golang/stackencrypt/plan/plantest/plantest_internal_test.go b/languages/golang/stackencrypt/plan/plantest/plantest_internal_test.go index db4d50c38..0996d0988 100644 --- a/languages/golang/stackencrypt/plan/plantest/plantest_internal_test.go +++ b/languages/golang/stackencrypt/plan/plantest/plantest_internal_test.go @@ -101,19 +101,19 @@ func TestRenameWithoutAPinIsAContextChange(t *testing.T) { _, err := check(path, individualV2(), m, false, "RERUN") mustContain(t, err, "CONTEXT CHANGES", - `column medicare_number: no field writes its context "individuals/medicare_number" any more.`, - `Field medicare_no (MedicareNo) now writes column medicare_no under "individuals/medicare_no"`, + `column medicare_number: no field writes its context ["individuals", "medicare_number"] any more.`, + `Field medicare_no (MedicareNo) now writes column medicare_no under ["individuals", "medicare_no"]`, `Pinning the rule that decides field medicare_no (MedicareNo) with plan.Column("medicare_number") keeps it.`, "RERUN", - "- context individuals/medicare_number", - "+ context individuals/medicare_no", + "- context [\"individuals\", \"medicare_number\"]", + "+ context [\"individuals\", \"medicare_no\"]", ) // A context change is data loss, not a migration. The new column is // still listed, since the guess may be wrong. if strings.Contains(err.Error(), "TARGET CHANGES") { t.Errorf("a rename is reported as a migration:\n%s", err) } - mustContain(t, err, "OTHER CHANGES", "new column medicare_no, under \"individuals/medicare_no\", with terms [eq]. It is also named above as a possible rename of column medicare_number.") + mustContain(t, err, "OTHER CHANGES", `new column medicare_no, under ["individuals", "medicare_no"], with terms [eq]. It is also named above as a possible rename of column medicare_number.`) pinned := plan.ForMessage(nil, "individuals", plan.FirstOf( plan.When(plan.Field("medicare_no"), plan.Encrypt(plan.EQL(se.Equality)), plan.Column("medicare_number")), @@ -168,13 +168,13 @@ func TestChangesAreSortedByWhatTheyCost(t *testing.T) { before: plan.ForMessage(nil, "individuals", plan.FirstOf(plan.When(plan.Field("medicare_number"), gov, plan.Identity("medicare_no"))).OrElse(base)), after: plan.ForMessage(nil, "individuals", base), section: "CONTEXT CHANGES", - says: []string{`column medicare_number: its context is "individuals/medicare_number", was "individuals/medicare_no".`, `with plan.Identity("medicare_no") keeps it.`}, + says: []string{`column medicare_number: its context is ["individuals", "medicare_number"], was ["individuals", "medicare_no"].`, `with plan.Identity("medicare_no") keeps it.`}, }, "table changed": { before: plan.ForMessage(nil, "individuals", base), after: plan.ForMessage(nil, "people", base), section: "CONTEXT CHANGES", - says: []string{`the message's table is people, was individuals.`, `restore plan.Table("individuals")`, `its context is "people/email", was "individuals/email". Restoring plan.Table("individuals") brings it back.`}, + says: []string{`the message's table is people, was individuals.`, `restore plan.Table("individuals")`, `its context is ["people", "email"], was ["individuals", "email"]. Restoring plan.Table("individuals") brings it back.`}, }, "table changed, nothing encrypted": { before: plan.ForMessage(nil, "individuals", plan.When(category.Present(), plan.Plaintext())), @@ -192,9 +192,9 @@ func TestChangesAreSortedByWhatTheyCost(t *testing.T) { before: plan.ForMessage(nil, "individuals", plan.FirstOf(plan.When(plan.Field("medicare_number"), plan.Encrypt(plan.Custom("individuals/medicare_number")))).OrElse(base)), after: plan.ForMessage(nil, "people", plan.FirstOf(plan.When(plan.Field("medicare_number"), plan.Encrypt(plan.Custom("people/medicare_number")))).OrElse(base)), section: "CONTEXT CHANGES", - says: []string{`column medicare_number: its context is "people/medicare_number", was "individuals/medicare_number". No plan.Column or plan.Identity pin`}, + says: []string{`column medicare_number: its context is ["people", "medicare_number"], was ["individuals", "medicare_number"]. No plan.Column or plan.Identity pin`}, // Restoring the table leaves the Custom context as it is. - never: []string{`was "individuals/medicare_number". Restoring`}, + never: []string{`was ["individuals", "medicare_number"]. Restoring`}, }, "table changed and field renamed": { before: plan.ForMessage(nil, "individuals", base), @@ -204,18 +204,14 @@ func TestChangesAreSortedByWhatTheyCost(t *testing.T) { also: []string{"OTHER CHANGES"}, says: []string{`Restoring plan.Table("individuals") and pinning the rule that decides field medicare_no (MedicareNo) with plan.Column("medicare_number") brings it back.`}, }, - // The Custom text reads like the EQL identity, but it is one text - // part, not the (table, column) pair: a different context. - "target kind changed, the context spelled the same": { + // The Custom label reads like the EQL identity, and it is the same + // context: both bind the label (table, column). Only the target + // changed, and nothing written is lost. + "target kind changed, the context the same": { before: plan.ForMessage(nil, "individuals", base), after: plan.ForMessage(nil, "individuals", plan.FirstOf(plan.When(plan.Field("medicare_number"), plan.Encrypt(plan.Custom("individuals/medicare_number", se.Equality)))).OrElse(base)), - section: "CONTEXT CHANGES", - also: []string{"OTHER CHANGES"}, - says: []string{ - `column medicare_number: its context is "individuals/medicare_number" (one Custom text part), was "individuals/medicare_number" (the EQL table/column pair).`, - "column medicare_number: its target is Custom, was EQL.", - }, - never: []string{"under the same context"}, + section: "OTHER CHANGES", + says: []string{"column medicare_number: its target is Custom, was EQL, under the same context."}, }, "target kind and context changed": { before: plan.ForMessage(nil, "individuals", base), @@ -223,7 +219,7 @@ func TestChangesAreSortedByWhatTheyCost(t *testing.T) { section: "CONTEXT CHANGES", also: []string{"OTHER CHANGES"}, says: []string{ - `column medicare_number: its context is "elsewhere/v1", was "individuals/medicare_number".`, + `column medicare_number: its context is ["elsewhere", "v1"], was ["individuals", "medicare_number"].`, "column medicare_number: its target is Custom, was EQL.", }, never: []string{"under the same context"}, @@ -232,13 +228,13 @@ func TestChangesAreSortedByWhatTheyCost(t *testing.T) { before: plan.ForMessage(nil, "individuals", plan.FirstOf(plan.When(plan.Field("medicare_number"), plan.Encrypt(plan.Custom("gov/v1")))).OrElse(base)), after: plan.ForMessage(nil, "individuals", plan.FirstOf(plan.When(plan.Field("medicare_number"), plan.Encrypt(plan.Custom("gov/v2")))).OrElse(base)), section: "CONTEXT CHANGES", - says: []string{`its context is "gov/v2", was "gov/v1". No plan.Column or plan.Identity pin`, "a plan.Custom context"}, + says: []string{`its context is ["gov", "v2"], was ["gov", "v1"]. No plan.Column or plan.Identity pin`, "a plan.Custom context"}, }, "encrypted now plaintext": { before: plan.ForMessage(nil, "individuals", base), after: plan.ForMessage(nil, "individuals", plan.FirstOf(plan.When(plan.Field("medicare_number"), plan.Plaintext())).OrElse(base)), section: "TARGET CHANGES", - says: []string{"column medicare_number is now field medicare_number, decided Plaintext.", `ciphertexts under "individuals/medicare_number"`}, + says: []string{"column medicare_number is now field medicare_number, decided Plaintext.", `ciphertexts under ["individuals", "medicare_number"]`}, }, "terms changed": { before: plan.ForMessage(nil, "individuals", base), @@ -250,7 +246,7 @@ func TestChangesAreSortedByWhatTheyCost(t *testing.T) { before: plan.ForMessage(nil, "individuals", base), after: plan.ForMessage(nil, "individuals", plan.FirstOf(plan.When(plan.Field("country"), plan.Encrypt(plan.EQL()))).OrElse(base)), section: "TARGET CHANGES", - says: []string{`field country, decided Plaintext before, is now encrypted into column country under "individuals/country".`}, + says: []string{`field country, decided Plaintext before, is now encrypted into column country under ["individuals", "country"].`}, }, "database column renamed": { before: plan.ForMessage(nil, "individuals", base), @@ -267,7 +263,7 @@ func TestChangesAreSortedByWhatTheyCost(t *testing.T) { section: "CONTEXT CHANGES", also: []string{"OTHER CHANGES"}, says: []string{ - `column medicare_num: no field writes its context "individuals/medicare_id" any more.`, + `column medicare_num: no field writes its context ["individuals", "medicare_id"] any more.`, `with plan.Column("medicare_num"), plan.Identity("medicare_id") keeps it.`, }, }, @@ -283,7 +279,7 @@ func TestChangesAreSortedByWhatTheyCost(t *testing.T) { before: plan.ForMessage(nil, "individuals", base), after: plan.ForMessage(nil, "individuals", plan.FirstOf(plan.When(plan.Field("id"), plan.Encrypt(plan.EQL(se.Ore)))).OrElse(base)), section: "OTHER CHANGES", - says: []string{`new column id, under "individuals/id", with terms [ore].`}, + says: []string{`new column id, under ["individuals", "id"], with terms [ore].`}, }, "facts changed": { before: plan.ForMessage(nil, "individuals", base), @@ -351,7 +347,7 @@ func TestALostContextWithNoCandidate(t *testing.T) { plan.Fact{Field: "country", GoField: "Country", Number: 4, Annotations: classified("system.operations")}, ) _, err := check(path, gone, m, false, "RERUN") - mustContain(t, err, "CONTEXT CHANGES", `column medicare_number: no field writes its context "individuals/medicare_number" any more. If its field was renamed, pin the renamed field's rule with plan.Column("medicare_number")`) + mustContain(t, err, "CONTEXT CHANGES", `column medicare_number: no field writes its context ["individuals", "medicare_number"] any more. If its field was renamed, pin the renamed field's rule with plan.Column("medicare_number")`) } // Custom columns may share a context, so a new one under the context of @@ -368,8 +364,8 @@ func TestSharedCustomContextIsNotARename(t *testing.T) { plan.Fact{Field: "c", GoField: "C", Number: 3, Annotations: classified("user.content")}, ), m, false, "RERUN") mustContain(t, err, "OTHER CHANGES", - `column a is no longer written. Its context "pii/v1" is not lost, since columns b, c still write it`, - `new column c, under "pii/v1"`) + `column a is no longer written. Its context ["pii", "v1"] is not lost, since columns b, c still write it`, + `new column c, under ["pii", "v1"]`) for _, never := range []string{"RENAME COLUMN", "CONTEXT CHANGES"} { if strings.Contains(err.Error(), never) { t.Errorf("failure says %q:\n%s", never, err) @@ -393,12 +389,12 @@ func TestARenameGuessStatesTheOtherReading(t *testing.T) { ), m, false, "RERUN") mustContain(t, err, "CONTEXT CHANGES", - `column home_phone: no field writes its context "individuals/home_phone" any more.`, - `Field work_phone (WorkPhone) now writes column work_phone under "individuals/work_phone" with the same facts, so it may be the same field renamed:`, + `column home_phone: no field writes its context ["individuals", "home_phone"] any more.`, + `Field work_phone (WorkPhone) now writes column work_phone under ["individuals", "work_phone"] with the same facts, so it may be the same field renamed:`, `with plan.Column("home_phone") keeps it.`, "If work_phone is instead a new field and home_phone was removed, do not pin it: that would store two fields in column home_phone under one context.", "OTHER CHANGES", - `new column work_phone, under "individuals/work_phone", with terms [none]. It is also named above as a possible rename of column home_phone.`, + `new column work_phone, under ["individuals", "work_phone"], with terms [none]. It is also named above as a possible rename of column home_phone.`, ) if strings.Contains(err.Error(), "likely") { t.Errorf("the guess is stated as likely:\n%s", err) @@ -495,13 +491,13 @@ func TestSnapshotIsDeterministic(t *testing.T) { table individuals column email - context individuals/email + context ["individuals", "email"] target EQL terms eq match fact fides.data_categories user.contact.email column medicare_number - context individuals/medicare_number + context ["individuals", "medicare_number"] target EQL terms eq fact fides.data_categories user.government_id @@ -524,15 +520,12 @@ func TestSnapshotRoundTrips(t *testing.T) { if i%2 == 1 { kind = kindCustom } - // A plan never builds an empty context, and parse refuses one. - context := v - if context == "" { - context = "t/empty" - } - s.columns = append(s.columns, column{name: v + string(rune('a'+i)), context: context, kind: kind, terms: []string{"eq", "ore", "odd term"}, facts: []fact{{v, v}, {"k", v}}}) + // Each segment is quoted, whatever it holds; a label has at least + // two. + s.columns = append(s.columns, column{name: v + string(rune('a'+i)), context: shape([]string{"t", v}), kind: kind, terms: []string{"eq", "ore", "odd term"}, facts: []fact{{v, v}, {"k", v}}}) s.plaintext = append(s.plaintext, plain{field: v + string(rune('a'+i)), facts: []fact{{v, "x"}}}) } - s.columns = append(s.columns, column{name: "bare", context: "t/bare", kind: kindEQL}) + s.columns = append(s.columns, column{name: "bare", context: shape([]string{"t", "bare"}), kind: kindEQL}) for i := range s.columns { sortFacts(s.columns[i].facts) } @@ -609,7 +602,7 @@ func TestTextOnlyAndUnreadableSnapshots(t *testing.T) { // parse refuses what render never writes, so a damaged snapshot falls back // to the plain diff rather than a summary built on a misreading. func TestParseRejects(t *testing.T) { - const col = "\ncolumn email\n context individuals/email\n target EQL\n terms eq\n" + const col = "\ncolumn email\n context [\"individuals\", \"email\"]\n target EQL\n terms eq\n" for name, tc := range map[string]struct{ text, says string }{ "empty file": {"", "no table line"}, "header only": {header, "no table line"}, @@ -622,8 +615,23 @@ func TestParseRejects(t *testing.T) { header + "\ntable individuals\n" + strings.Replace(col, " target EQL\n", "", 1), `column "email" has no context or target line`, }, + // The spelling before contexts were segments: "notes/v1" there was + // one text part for a Custom column, so reading it as a label + // would guess at the context. + "context as joined text": { + header + "\ntable individuals\n" + strings.Replace(col, `["individuals", "email"]`, "individuals/email", 1), + "is not a list of segments", + }, + "context with a stray separator": { + header + "\ntable individuals\n" + strings.Replace(col, `"individuals", "email"`, `"individuals","email"`, 1), + `expected ", " or "]"`, + }, + "context with trailing text": { + header + "\ntable individuals\n" + strings.Replace(col, `"email"]`, `"email"] x`, 1), + `expected ", " or "]"`, + }, "column without a context": { - header + "\ntable individuals\n" + strings.Replace(col, " context individuals/email\n", "", 1), + header + "\ntable individuals\n" + strings.Replace(col, " context [\"individuals\", \"email\"]\n", "", 1), `column "email" has no context or target line`, }, } { diff --git a/languages/golang/stackencrypt/plan/plantest/snapshot.go b/languages/golang/stackencrypt/plan/plantest/snapshot.go index 3591dac79..31b9c26ab 100644 --- a/languages/golang/stackencrypt/plan/plantest/snapshot.go +++ b/languages/golang/stackencrypt/plan/plantest/snapshot.go @@ -30,12 +30,17 @@ type snapshot struct { // column is one encrypted field, named by its record key. type column struct { - name string + name string + // context is the label the column binds, spelled as its segments by + // [shape]: ["individuals", "email"]. The segments, not a joined text, + // are what a context is, so the file spells them: "notes/v1" as one + // text part and as two segments are different contexts that read alike. context string - // kind is how the context is bound: kindEQL when it is the column - // identity, "
/", so the table and a column rename move - // it; kindCustom when the target supplies it whatever the column, so - // several columns may share it. + // kind is how the context is chosen: kindEQL when it is the column + // identity, ["
", ""], so the table and a column rename + // move it; kindCustom when the target supplies it whatever the column, + // so several columns may share it. Both bind a label, so an EQL column + // and a Custom one with the same segments bind the same context. kind string terms []string facts []fact @@ -85,33 +90,29 @@ func targetKind(t plan.Target) (string, error) { return kindEQL, nil } -// contextText is the text a snapshot stores for a column's context, the -// same text the policy spells it with. An EQL column's context is its -// identity's label, the pair (table, column identity), stored as -// "
/"; a Custom column's is the one text part -// [plan.Custom] was given, stored as written. The two are told apart by -// the column's target line, not by the text. The text is checked against -// the context the plan actually binds, so the file cannot name a context -// the plan does not use. +// contextText is the [shape] a snapshot stores for a column's context. An +// EQL column's context is its identity's label, (table, column identity); +// a Custom column's is the label [plan.Custom] parses from its argument. +// The shape is checked against the context the plan actually binds, so +// the file cannot name a context the plan does not use. func contextText(table string, kind string, d plan.Decision, fp stackencrypt.FieldPlan) (string, error) { - var text string - var want stackencrypt.Context + var label stackencrypt.Label switch kind { case kindEQL: identity := d.Identity() if identity == "" { identity = fp.Name } - label, err := plan.Identifier{Table: table, Column: identity}.Label() - if err != nil { + var err error + if label, err = (plan.Identifier{Table: table, Column: identity}).Label(); err != nil { return "", fmt.Errorf("plantest: column %q: %w", fp.Name, err) } - text, want = label.String(), label.Context() default: // A Custom context has no accessor: plan.Custom renders its // argument quoted, as Custom("", ...), so read it back. target, _ := d.Target() spelled, ok := strings.CutPrefix(fmt.Sprint(target), "Custom(") + var text string var err error if ok { if text, err = strconv.QuotedPrefix(spelled); err == nil { @@ -121,22 +122,67 @@ func contextText(table string, kind string, d plan.Decision, fp stackencrypt.Fie if !ok || err != nil { return "", fmt.Errorf("plantest: column %q: cannot spell the context of target %v; a target that does not bind its column identity must be plan.Custom", fp.Name, target) } - var l stackencrypt.Label - if l, err = stackencrypt.ParseLabel(text); err != nil { + if label, err = stackencrypt.ParseLabel(text); err != nil { return "", fmt.Errorf("plantest: column %q: %w", fp.Name, err) } - want = l.Context() } - if !want.Equal(fp.Context) { - return "", fmt.Errorf("plantest: column %q: the plan binds a context other than %q", fp.Name, text) + if !label.Context().Equal(fp.Context) { + return "", fmt.Errorf("plantest: column %q: the plan binds a context other than %s", fp.Name, shape(label.Segments())) + } + return shape(label.Segments()), nil +} + +// shape spells a label's segments as a snapshot stores them: a bracketed, +// comma-separated list of Go string literals, ["individuals", "email"]. +// Every segment is quoted, so the spelling does not depend on which +// characters a segment holds. +func shape(segments []string) string { + quoted := make([]string, len(segments)) + for i, s := range segments { + quoted[i] = strconv.Quote(s) + } + return "[" + strings.Join(quoted, ", ") + "]" +} + +// segmentsOf reads a [shape] back into its segments. It accepts only what +// shape writes, so a context in any other spelling (a file written before +// contexts were spelled as segments) is refused rather than guessed at. +func segmentsOf(spelled string) ([]string, error) { + rest, ok := strings.CutPrefix(spelled, "[") + if !ok { + return nil, fmt.Errorf("context %s is not a list of segments such as [\"table\", \"column\"]; a file written by an older plantest spells contexts differently, so read what the new one writes before recording it", spelled) + } + var out []string + for { + q, err := strconv.QuotedPrefix(rest) + if err != nil { + return nil, fmt.Errorf("context %s: bad segment", spelled) + } + seg, _ := strconv.Unquote(q) + out = append(out, seg) + rest = rest[len(q):] + if r, ok := strings.CutPrefix(rest, ", "); ok { + rest = r + continue + } + if rest != "]" { + return nil, fmt.Errorf("context %s: expected \", \" or \"]\" after a segment", spelled) + } + if shape(out) != spelled { + return nil, fmt.Errorf("context %s is not spelled as plantest writes it", spelled) + } + return out, nil } - return text, nil } // contextOf is the context a snapshot's column names, rebuilt from its -// text and target kind: what [contextText] wrote. +// segments: what [contextText] wrote. func contextOf(c column) (stackencrypt.Context, error) { - label, err := stackencrypt.ParseLabel(c.context) + segments, err := segmentsOf(c.context) + if err != nil { + return stackencrypt.Context{}, err + } + label, err := stackencrypt.NewLabel(segments...) if err != nil { return stackencrypt.Context{}, err } @@ -248,7 +294,7 @@ func (s snapshot) render() []byte { fmt.Fprintf(&b, "\ntable %s\n", token(s.table)) for _, c := range s.columns { fmt.Fprintf(&b, "\ncolumn %s\n", token(c.name)) - fmt.Fprintf(&b, " context %s\n", token(c.context)) + fmt.Fprintf(&b, " context %s\n", c.context) fmt.Fprintf(&b, " target %s\n", c.kind) fmt.Fprintf(&b, " terms %s\n", termList(c.terms)) writeFacts(&b, c.facts) @@ -317,6 +363,15 @@ func parse(data []byte) (snapshot, error) { continue } indented := strings.HasPrefix(line, " ") + // A context line holds a shape, not tokens: its segments are + // quoted and separated by ", ". + if spelled, ok := strings.CutPrefix(line, " context "); ok && colAt >= 0 { + if _, err := segmentsOf(spelled); err != nil { + return snapshot{}, fmt.Errorf("line %d: %w", n+1, err) + } + s.columns[colAt].context = spelled + continue + } toks, err := tokens(strings.TrimSpace(line)) if err != nil { return snapshot{}, fmt.Errorf("line %d: %w", n+1, err) @@ -330,8 +385,6 @@ func parse(data []byte) (snapshot, error) { case !indented && toks[0] == "plaintext" && len(toks) == 2: s.plaintext = append(s.plaintext, plain{field: toks[1]}) colAt, plainAt = -1, len(s.plaintext)-1 - case indented && toks[0] == "context" && len(toks) == 2 && colAt >= 0: - s.columns[colAt].context = toks[1] case indented && toks[0] == "target" && len(toks) == 2 && colAt >= 0 && (toks[1] == kindEQL || toks[1] == kindCustom): s.columns[colAt].kind = toks[1] case indented && toks[0] == "terms" && len(toks) >= 2 && colAt >= 0: diff --git a/languages/golang/stackencrypt/plan/plantest/testdata/TestPolicies/individuals.golden b/languages/golang/stackencrypt/plan/plantest/testdata/TestPolicies/individuals.golden index 566d6ee16..fa5a7bbff 100644 --- a/languages/golang/stackencrypt/plan/plantest/testdata/TestPolicies/individuals.golden +++ b/languages/golang/stackencrypt/plan/plantest/testdata/TestPolicies/individuals.golden @@ -5,25 +5,25 @@ table individuals column email - context individuals/email + context ["individuals", "email"] target EQL terms eq match fact fides.data_categories user.contact.email column medicare_num - context individuals/medicare_number + context ["individuals", "medicare_number"] target EQL terms eq fact fides.data_categories user.government_id column name - context individuals/name + context ["individuals", "name"] target EQL terms ore fact fides.data_categories user.name column notes - context individuals-notes/v1 + context ["individuals-notes", "v1"] target Custom terms none fact fides.data_categories user.content diff --git a/packages/stack-encrypt/CHANGELOG.md b/packages/stack-encrypt/CHANGELOG.md index 8f0ecf0a6..e27aeac6f 100644 --- a/packages/stack-encrypt/CHANGELOG.md +++ b/packages/stack-encrypt/CHANGELOG.md @@ -48,7 +48,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 indexed fields under one label are refused for the same reason the builder refuses them (`PlanError::SharedIdentity`). A record a Go program wrote with a `label=` tag is unaffected; one written with a `context=` - tag is not readable through a plan. + tag is not readable through a plan. For the same reason Go + `plan.Custom("notes/v1")` now binds the label `["notes", "v1"]`, not the + one text part `"notes/v1"`: a row written under the old form does not + decrypt and its terms match no query, with no error, and a one-segment + `Custom("ctx")` is refused. `plantest` snapshots now spell every context + as its segments (`context ["notes", "v1"]`), so a golden file changes on + regeneration and refuses the old spelling until it does; check each + `target Custom` column before recording it. - **Every data plan field seals the tagged `FfiValue` leaf, whatever its `"type"`**, as every field did before; the type admits indexes and checks kinds and changes no bytes, so a row written without a type opens under a From 0713d1d81c4d0f1a666b1e5d8e56c1117b44010a Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Tue, 6 Oct 2026 21:05:22 -0700 Subject: [PATCH 14/14] docs(stack-encrypt): plan says a binding's rows need dynamic::Value fields The "binding lowers into the same builder" section said the data path is not a second executor, which is true of the engine and not of the stored bytes: a data plan seals the tagged Value leaf, and a bare String reader opens it as "\nalice" with no error. A developer writing a Rust chain reads this module, not dynamic::record, so the rule and #1118 are named here. --- packages/stack-encrypt/src/plan/mod.rs | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/packages/stack-encrypt/src/plan/mod.rs b/packages/stack-encrypt/src/plan/mod.rs index 9b211fc50..aca64709a 100644 --- a/packages/stack-encrypt/src/plan/mod.rs +++ b/packages/stack-encrypt/src/plan/mod.rs @@ -269,6 +269,16 @@ //! and the plan's opener. It is the third author of this grammar, beside a //! Rust chain and the derive, and not a second executor. //! +//! The engine is shared; the stored leaf is not. A data plan seals every +//! field as the tagged `dynamic::Value` leaf, whatever its type, so a Rust +//! record whose rows a binding also reads must declare those fields as +//! `dynamic::Value`. A bare `String` or `u32` field derives the same terms +//! but encrypts a different leaf. A bare `String` reader accepts a +//! binding's leaf and returns its type tag as a leading line feed +//! (`"\nalice"`), with no error. One leaf encoding for both, or the +//! encoding bound into the leaf's context so the wrong reader fails, is +//! tracked in #1118. +//! //! # How a chain lowers //! //! The builder adds no cryptographic operation and no executor; every call