Repository navigation
docs: the Go SDK design, and the language SDK design principles #1070
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
auxesis
merged 22 commits into
docs/plan-go-encrypt-as
from
docs/plan-builder-idiomatic-go
Oct 6, 2026
Merged
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
9c3277f
docs(plans): complete Go examples for the plan builder
auxesis 7bcf794
docs(plans): Go binding as typed plans with ctx-first methods
auxesis 5640741
docs(plans): generate the encrypted Go type with stashgen
auxesis 3c44aab
docs(plans): design stashgen, and check storage structs at compile time
auxesis 1d67487
docs(plans): run the policy at generate time; no Must for a program t…
auxesis b946e2c
docs(plans): no panic left in the Go binding; stashrt becomes gensupport
auxesis 3ce9129
docs(plans): one Encrypt over a slice; usage steps before the stashge…
auxesis e4f9697
docs(plans): say what GoField is in the policy example
auxesis a30b9d2
docs: language SDK design principles, with ADR-0002
auxesis edcba1a
docs(plans): apply the SDK principles to the Go SDK design
auxesis 6622c38
docs(plans): name the model flag -model
auxesis 29446f6
docs(plans): one Batch that writes to destinations
auxesis f11111e
docs(plans): EQL type names from the catalog, and what works today
auxesis 0e73334
docs(plans): the verified status of each EQL type; eql is a package
auxesis 11a174c
docs(plans): name the Go package encrypt
auxesis 40d93cc
docs(plans): name the Go credentials package auth
auxesis 445c0a8
docs(plans): keep the policy, for types a schema generates
auxesis 75b422d
docs(plans): settle the review's conflicts with the base plan
auxesis 2000cad
docs(plans): one call covers one type, and a whole value has a declar…
auxesis 677f663
docs(adr): amend ADR-0004 to ADR-0007 for the Go SDK
auxesis 9a77dec
docs(plans): apply Dan's decisions of 2026-10-05 to the Go SDK design
auxesis 48342ab
docs(plans): generated examples build the encrypted type from what th…
auxesis File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| # Go examples for the plan builder | ||
|
|
||
| These files show the Go SDK from [the plan builder design](../2026-10-04-plan-builder.md#the-go-sdk) as complete programs. | ||
| The SDK does not exist yet, so this code does not build in this repository. | ||
| The module path is `example.com/app`. | ||
|
|
||
| ## What each file shows | ||
|
|
||
| | File | What it shows | | ||
| |---|---| | ||
| | [`main.go`](main.go) | A client, one cipher for each tenant, and a batch of two types in one request | | ||
| | [`users/model.go`](users/model.go) | A struct with `stash` tags, and the `go:generate` line | | ||
| | [`users/user_stash.go`](users/user_stash.go) | The file `stashgen` writes: the encrypted type, `Encrypt`, `Decrypt` and `Fields` | | ||
| | [`users/sqlstore.go`](users/sqlstore.go) | `database/sql`: batch insert in a transaction, and a search by email | | ||
| | [`users/gormstore.go`](users/gormstore.go) | GORM, with the generated type as the model | | ||
| | [`users/sqlcstore.go`](users/sqlcstore.go) | sqlc, with its row struct converted to the generated type | | ||
| | [`sqlc/`](sqlc/) | The schema, the queries and the overrides that generate [`internal/userdb/`](internal/userdb/) | | ||
| | [`accounts/account.go`](accounts/account.go) | An embedded `gorm.Model`, another library's tags, an unexported field and `-redact` | | ||
| | [`contacts/contacts.go`](contacts/contacts.go) | [`crm.Contact`](crm/contact.go), a type in another package, in separate columns, with a model | | ||
| | [`documents/documents.go`](documents/documents.go) | An `opaque` struct, sealed as one value | | ||
| | [`proto/`](proto/) | A protobuf message whose fields carry data categories, which generates [`internal/pb/`](internal/pb/) | | ||
| | [`rules/rules.go`](rules/rules.go) | Rules that decide what to encrypt from each field's data categories | | ||
| | [`cmd/genencrypt/main.go`](cmd/genencrypt/main.go) | The generate program that runs the rules | | ||
| | [`individuals/`](individuals/) | The file the rules give for the protobuf message, and a store that uses it | | ||
|
|
||
| The `users` example uses `TextEq`, which is the one EQL type the engine produces today. | ||
|
|
||
| ## What was checked | ||
|
|
||
| | Claim | Status | | ||
| |---|---| | ||
| | Every Go file type-checks | Run. `go vet ./...` passes against a stub of the SDK. The stub is not in this repository, and it has signatures only. | | ||
| | The policy example compiles against real protobuf code | Run. buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. | | ||
| | A field added to the protobuf message does not stop the build | Run. It builds, as the plan says: CI finds that change. | | ||
| | The protobuf source reads the field options, and the rules run | Not run. Neither the source nor the generator exists. | | ||
| | A change to a tagged struct, a model or a type in another package stops the build | Run. Each change fails `go build` with "cannot convert". | | ||
| | sqlc's row struct converts to the generated type | Run. The conversions in `users/sqlcstore.go` compile against real sqlc output. | | ||
| | A struct from another package with an unexported field cannot convert | Run, with `sync.Once`. | | ||
| | A generated file from another version does not compile | Run. A file that names an unknown version constant fails `go build`. | | ||
| | The files `stashgen` writes | Not run. `stashgen` does not exist, and the five `_stash.go` files are written by hand. | | ||
| | The SDK can be built with these signatures | Not run. | | ||
| | The record fixture | Not run. It does not exist. | | ||
| | The guest returns an EQL value for a field with `encrypt_into` | Not run. The target form and the dispatch do not exist. | | ||
| | `stashgen` checks a declaration with the embedded guest | Not run. | | ||
| | The code works with a database, GORM or pgx | Not run. Nothing here has connected to a database. | | ||
| | The EQL types, and the value the guest returns for them | Not run. `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. | | ||
| | The steps in "Use the SDK" | Not run. Nobody has followed them. | | ||
|
|
||
| ## Generate the protobuf package | ||
|
|
||
| The protobuf package was generated with buf v1.50.0 and `protoc-gen-go`. | ||
| Run `buf generate` in `proto/` to generate it again. | ||
|
|
||
| ## Generate the sqlc package | ||
|
|
||
| The sqlc package was generated with sqlc v1.31.1. | ||
| Run `sqlc generate` in `sqlc/` to generate it again. | ||
|
|
||
| ## Use sqlc with EQL columns | ||
|
|
||
| Three rules apply when a column has an EQL type: | ||
|
|
||
| - **Give sqlc a file that declares the domains.** | ||
| sqlc cannot parse the EQL install bundle, because it rejects function overloads that differ only by `text` and `text[]`. | ||
| The file [`sqlc/eql-domains.sql`](sqlc/eql-domains.sql) declares each domain as `jsonb`, and the database still gets the real bundle from `stash eql install`. | ||
| - **Spell a `db_type` override exactly as the schema spells the type.** | ||
| `public.eql_v3_text_eq` and `eql_v3_text_eq` are two different spellings to sqlc. | ||
| An override with the other spelling matches nothing, and the column becomes `interface{}`. | ||
| - **Cast a query parameter once, straight to the query domain.** | ||
| sqlc types a parameter by its first cast. | ||
| `$1::eql_v3.query_text_eq` gets the override type, and `$1::jsonb::eql_v3.query_text_eq` gets `json.RawMessage`. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| // Package accounts shows an embedded struct, another library's tags, an | ||
| // unexported field, and generated print methods. | ||
| package accounts | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| "github.com/cipherstash/stack/languages/golang/encrypt" | ||
| "gorm.io/gorm" | ||
| ) | ||
|
|
||
| //go:generate go tool stashgen -type Account -redact | ||
|
|
||
| type Account struct { | ||
| _ struct{} `stash:"context=accounts"` | ||
|
|
||
| // One tag decides for every field of gorm.Model, which cannot carry tags. | ||
| gorm.Model `stash:",passthrough"` | ||
|
|
||
| // stashgen copies the gorm and json tags onto EncryptedAccount. | ||
| Email string `stash:"email,encrypt_into=TextEq" gorm:"uniqueIndex" json:"email"` | ||
|
|
||
| // An unexported field with no stash tag is ignored, and stashgen and the | ||
| // running program both warn about it. | ||
| cache string | ||
|
|
||
| // The tag acknowledges the skip, so nothing warns. | ||
| token string `stash:"-"` | ||
| } | ||
|
|
||
| func (EncryptedAccount) TableName() string { return "accounts" } | ||
|
|
||
| func Create(ctx context.Context, db *gorm.DB, cipher *encrypt.Cipher, accounts []Account) error { | ||
| encrypted, err := Encrypt(ctx, cipher, accounts) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| return db.WithContext(ctx).Create(&encrypted).Error | ||
| } |
151 changes: 151 additions & 0 deletions
151
docs/plans/2026-10-04-plan-builder/accounts/account_stash.go
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| // Command genencrypt runs the rules at go generate time. | ||
| package main | ||
|
|
||
| import ( | ||
| "log" | ||
|
|
||
| "example.com/app/rules" | ||
| "github.com/cipherstash/stack/languages/golang/encrypt/policy/protosource" | ||
| "github.com/cipherstash/stack/languages/golang/stashgen" | ||
| ) | ||
|
|
||
| func main() { | ||
| err := stashgen.Generate(protosource.New(), rules.Individuals, | ||
| stashgen.Output("../individuals/individual_stash.go")) | ||
| if err != nil { | ||
| log.Fatal(err) | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| // Package contacts encrypts crm.Contact, a type from another package, into | ||
| // separate columns: one for the ciphertext and one for each term. | ||
| package contacts | ||
|
|
||
| import ( | ||
| "context" | ||
| "database/sql" | ||
|
|
||
| "example.com/app/crm" | ||
| "github.com/cipherstash/stack/languages/golang/encrypt" | ||
| "gorm.io/gorm" | ||
| ) | ||
|
|
||
| //go:generate go tool stashgen -type contactStash -for crm.Contact -model Rows=ContactRow | ||
|
|
||
| // contactStash declares the tags for crm.Contact, which cannot carry them. | ||
| // stashgen matches each field to the crm.Contact field with the same name and | ||
| // type, and refuses a crm.Contact field this struct does not name. | ||
| type contactStash struct { | ||
| _ struct{} `stash:"context=contacts"` | ||
| ID int64 `stash:"id,passthrough"` | ||
| Email string `stash:"email,encrypt,index=equality;match"` | ||
| PhoneNumber string `stash:"phone_number,encrypt,index=equality"` | ||
| Internal string `stash:"-"` | ||
| } | ||
|
|
||
| // ContactRow is a model: one field for each column. Each tag names the output | ||
| // the field holds. | ||
| type ContactRow struct { | ||
| ID int64 `stash:"id"` | ||
| Email encrypt.Ciphertext `stash:"email"` | ||
| EmailEq encrypt.EqualityTerm `stash:"email,equality"` | ||
| EmailMatch encrypt.MatchTerm `stash:"email,match"` | ||
| PhoneNumber encrypt.Ciphertext `stash:"phone_number"` | ||
| PhoneNumberEq encrypt.EqualityTerm `stash:"phone_number,equality"` | ||
| } | ||
|
|
||
| func (ContactRow) TableName() string { return "contacts" } | ||
|
|
||
| // Create writes with database/sql and passes each output itself. | ||
| func Create(ctx context.Context, db *sql.DB, cipher *encrypt.Cipher, list []crm.Contact) error { | ||
| encrypted, err := Encrypt(ctx, cipher, list) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| for _, e := range encrypted { | ||
| _, err := db.ExecContext(ctx, ` | ||
| INSERT INTO contacts (id, email, email_eq, email_match, phone_number, phone_number_eq) | ||
| VALUES ($1, $2, $3, $4, $5, $6)`, | ||
| e.ID, e.Email.Ciphertext, e.Email.Equality, e.Email.Match, | ||
| e.PhoneNumber.Ciphertext, e.PhoneNumber.Equality) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // CreateWithGORM writes the model, which GORM maps one field to one column. | ||
| func CreateWithGORM(ctx context.Context, db *gorm.DB, cipher *encrypt.Cipher, list []crm.Contact) error { | ||
| rows, err := EncryptRows(ctx, cipher, list) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| return db.WithContext(ctx).Create(&rows).Error | ||
| } | ||
|
|
||
| func IDByPhone(ctx context.Context, db *sql.DB, cipher *encrypt.Cipher, phone string) (int64, error) { | ||
| term, err := Fields.PhoneNumber.Equality(ctx, cipher, phone) | ||
| if err != nil { | ||
| return 0, err | ||
| } | ||
| var id int64 | ||
| err = db.QueryRowContext(ctx, `SELECT id FROM contacts WHERE phone_number_eq = $1`, term).Scan(&id) | ||
| return id, err | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Correct the
main.godescription.The table says
main.gosends "a batch of two types in one request".main.goLine 61 says "One call for each type, and one ZeroKMS request for each call". The PR objectives also defer multi-type batching. As a result, the README describes behavior that the example no longer shows.Proposed fix
📝 Committable suggestion
🤖 Prompt for AI Agents