feat(mail): manifest-declared mail.env value maps for enum-style SMTP config (#302)

docs/dev/capabilities.yml

@@ -29,7 +29,7 @@
 
 # Bumped whenever the capabilities list changes, so a consumer can tell how current
 # the manifest it validated against was (the published catalog stamps this number).
-version: 1
+version: 2
 
 capabilities:
   - id: secret-injection
@@ -89,3 +89,14 @@ capabilities:
       Per-app operator-supplied configuration. A manifest `config:` block renders a
       form at install and a Settings editor on the app page, injecting each answer
       under its own env-var name into the target service's compose override.
+
+  - id: mail-enable-enum
+    since: "0.x"
+    ref: "#302; APP_MANIFEST.md # D3; SERVICE_PROVISIONING.md # BYO outgoing mail"
+    summary: >
+      Manifest-declared mail.env value maps. An app whose mail config is a
+      boot-validated enum (twenty's EMAIL_DRIVER, vaultwarden's SMTP_SECURITY)
+      declares, per app-owned var, the token for each mail state (from: bound |
+      encryption); the brain resolves and stamps the final value under the app's
+      own name in both bound and unbound states, where compose interpolation can't
+      remap one enum's tokens onto another's.

docs/dev/catalog-import-gaps.md

@@ -208,7 +208,7 @@ Unlike `docs/progress/` entries (frozen ADR snapshots), this file is **mutable b
 - **Trigger:** `EMAIL_DRIVER` — Twenty's outbound-mail switch is a **boot-validated enum** (`logger` | `smtp`, default `logger`), not a boolean. The established malmo BYO-mail compose gate (`${MALMO_MAIL_HOST:+smtp}`, as `gitea`/`kimai`/`paperless-ngx` use to turn mail on only when a provider is bound) expands to the empty string on an unbound box, and `EMAIL_DRIVER=""` fails Twenty's config validation at startup — verified live: the container exits 1 with `EMAIL_DRIVER must be one of the following values` / `Config variables validation failed`.
 - **What breaks:** malmo's registered SMTP provider can't be auto-injected into Twenty by env the way it is for boolean-gated apps. Mail itself is not broken — the admin enables it in-app under **Settings → Emails** (Twenty's DB-backed config layer, which overrides the env default), so invitations and notifications send after one manual step — but the "it just works from the box's mail provider" auto-wire that `mail: {optional: true}` + `MALMO_MAIL_*` gives other apps does not apply, so the manifest declares no `mail:` binding.
 - **Why malmo can't satisfy it (v1):** the BYO-mail injection (#122) is shaped for a boolean/`:+`-gated enable flag; an enum driver that boot-rejects empty-string can't be gated by a single compose expansion (there is no `smtp`-when-bound / `logger`-when-unbound substitution). A malmo-side fix would be injecting a "mail-bound" boolean or a driver sentinel (e.g. `MALMO_MAIL_DRIVER`) the compose could map onto the enum's two valid values.
-- **Status:** open — shipped degraded (`catalog/twenty`); mail configured in-app, the same path `listmonk` takes for its DB-stored SMTP settings. New gap-class: recurs for any app whose mail-enable is a boot-validated enum rather than a boolean.
+- **Status:** implemented (#302 — manifest `mail.env` value maps, `APP_MANIFEST.md` # D3, `SERVICE_PROVISIONING.md` # BYO outgoing mail). The manifest declares, per app-owned mail env var, the token for each mail state (`from: bound|encryption`) and the brain resolves + stamps the final value in Go — closing both this enable-enum facet (twenty's `EMAIL_DRIVER`) and the security-enum facet (vaultwarden's `SMTP_SECURITY`), neither of which compose interpolation can remap. Emitted in both bound and unbound states so the enum boots valid unbound (killing the empty-string boot-reject the `:+` gate caused). Recorded in [`capabilities.yml`](capabilities.yml) under id `mail-enable-enum`. The brain mechanism ships here; wiring `catalog/twenty`'s manifest (`EMAIL_DRIVER`) and the store repo's `apps/vaultwarden` (`SMTP_SECURITY`) plus the live send check are the coordinated catalog/store follow-up.
 
 ### nonroot-data-ownership — formbricks (2026-06-20)
 

docs/progress/README.md

@@ -202,3 +202,4 @@ Oldest first; append new entries to the bottom.
 | [reconcile-pending-recreate.md](reconcile-pending-recreate.md) — Make the reconcile pass converge env-restamping drift on a **running** container (**closes #268**), the gap `manifest-config-block.md` (#264) documented and `RebindMail` shared. `SetConfig`/`RebindMail` are brain-commits-first: store + override/.env, then `compose up -d` a running instance. A failed `compose up` left a container that **kept running** on its old env — reconcile re-created an already-running container only on resource-limit drift, never env drift — so it stayed stale until the user retried (a fallen-over container or brain restart already converged via the "no containers" branch). **Fix:** a `pending_recreate` boolean on `instances` (column + idempotent migration + `Instance.PendingRecreate` + `SetInstancePendingRecreate`). New `recreateRunning(ctx, inst)` helper wraps the running-instance edit's `compose up`: on failure it **marks** the instance pending (the committed override/.env is the reconstructible intent), on success it **clears** the marker; `SetConfig`/`RebindMail` route through it. `Reconcile`'s already-up branch now recreates when resource-limit policy drifted **or** the marker is set — one `compose up -d` converges both (env read at container-create) — and clears the marker on success, with the `restore()` rewind guarded to the resource-stanza patch while the marker (not a file rewind) makes the env recreate retryable; the "no containers" branch also clears the marker after a bring-up so a pending-and-fallen-over instance is satisfied without a redundant later recreate. `Start` also clears the marker on a successful recreate, so a Stop→Start cycle doesn't leave a stale marker for reconcile to redundantly retry (a self-review catch). Covers config + mail + any future env-restamping op in **one place** (not special-cased to config). `APP_LIFECYCLE.md`'s reconcile drift list gained the pending-recreate case as a fourth bullet. Startup-pass cadence unchanged (no timer; converges on next brain start) — matches the issue's "self-healing-on-restart edge" framing and the spec's no-reconciler-loop stance. `make check` green | done |
 | [hosted-grow-root-disk.md](hosted-grow-root-disk.md) — Grow the hosted image's root filesystem to fill the whole provider disk on boot. The image bakes a fixed **8 GiB** root (sparse raw stays small) but nothing grew it onto the far larger provider disk, so a box ran on ~8 GiB — docker image storage + the brain's SQLite store share that one volume, so a single app install can fill it and the brain's first store write 500s login (the box looks offline though it is up). **Fix:** a runtime `repart.d` definition (`Type=root`, `GrowFileSystem=yes`, no size cap) + a `malmo-grow-root.service` oneshot running `systemd-repart --dry-run=no`, ordered `Before=docker.service host-agent.service` and **fail-closed** (`Requires=` from both, via a docker drop-in and host-agent's unit) so the write-heavy services refuse to start on an un-grown root. Build-time root stays pinned at 8 GiB; the two repart definitions are deliberately separate. Lean set gains `systemd-repart` + `libfdisk1` only (no `libcryptsetup`, cryptsetup cut preserved). Boot-proof asserts the tool is present and the unit reached `active`; real full-disk growth is a provider-box acceptance step. **Known:** the stock `systemd-repart.service` also runs the same config (harmless double execution — idempotent; consolidation deferred pending a live re-test), and a pre-existing unrelated `/setup` 503 boot-proof failure reproduces on clean `main`, tracked separately | done |
 | [hosted-setup-boot-proof-race.md](hosted-setup-boot-proof-race.md) — Close the `/setup` 503 boot-proof failure [hosted-grow-root-disk.md](hosted-grow-root-disk.md) tracked separately, and confirm the wildcard-TLS/`:443` path is green. The `unseeded` boot polled `POST /api/v1/setup` for **403** but broke on any of `403\|503\|409\|200`, so a transient **503** — Caddy answering "no ready `/api` upstream" in the first second after the control-plane stack comes up, before the brain's listener + dashboard route land — ended the loop and failed the proof, though the box is correct (the brain returns 403 unconditionally on hosted, no 503 path; the diag showed all four containers "Up <1–2 seconds"). **Fix:** break only on a definitive `403\|409\|200` and ride through `502\|503`, exactly as the `/api/v1/me` poll above already does; a genuinely stuck `/setup` still fails after the 30s window. The louder "`:443` never binds" symptom was a **broken-build artifact, not a live regression**: the build-and-boot CI job (`CI / Cloud image`, `publish=false`) passed both the `unseeded` and `seeded` boots (`seeded` hard-asserts `:443` bound + `caddy: wildcard TLS configured`), and every known real-box root cause (`certificates.automate` #301, seed-fetch keep-alive, static resolver) is already fixed in-tree — so no product change. Adds `docs/dev/hosted-boot-proof.md` (runbook: happy-path flow, brain-log milestones, symptom→where-to-look, how to run) and de-stales `TESTING.md`'s `/setup` gate description (the as-built SSO/403, superseding the secret 401/200). Test-lane + docs only | done |
+| [mail-enum-value-maps.md](mail-enum-value-maps.md) — Generalize BYO-mail injection so an app whose mail config is a **boot-validated enum** auto-wires, where compose interpolation can't remap tokens (**closes #302**, extends [byo-outgoing-mail.md](byo-outgoing-mail.md) #122). Compose `environment:` can substitute / default / test-presence but not remap one enum onto another, so twenty's `EMAIL_DRIVER` (boot-rejects the empty string `${MALMO_MAIL_HOST:+smtp}` expands to unbound) and vaultwarden's `SMTP_SECURITY` (`off\|starttls\|force_tls` vs malmo's `none\|starttls\|tls`) stayed degraded. **Fix:** manifest `mail.env` declares, per app-owned var, the token for each mail state (`from: bound\|encryption` + a domain-covering `map`); the brain resolves the final token in Go (`mailAppEnvLines`) and stamps it under the app's own name — no `MALMO_` indirection, the same direct-stamp convention `config:` uses. `validateMail` reuses the config `app_env` guards (uppercase, never `MALMO_`/loader var) + requires the map cover its domain exactly. **Emitted in both bound and unbound states** (the load-bearing change) — unbound resolves the `none`/`unbound` tokens so the enum is present-and-valid and never boot-rejects; `rewriteEnvMail` strips the declared vars by name (not just the `MALMO_MAIL_` prefix) and re-resolves on rebind, no duplicate lines. Ledger `mail-enable-enum — twenty` → implemented; `capabilities.yml` gains id `mail-enable-enum` (version 1→2). Brain mechanism only — `catalog/twenty` + store-repo `apps/vaultwarden` manifest wiring + live send checks are the coordinated follow-up (catalog cut over to the store repo). `make check` green | done |

docs/progress/mail-enum-value-maps.md

@@ -0,0 +1,39 @@
+# Mail enum value maps — manifest-declared `mail.env`, brain-resolved tokens
+
+- **Status:** done
+- **Date:** 2026-07-08
+- **Specs touched:** `APP_MANIFEST.md` (# D3 — new `mail.env` subsection), `SERVICE_PROVISIONING.md` (# BYO outgoing mail — enum-config paragraph), `docs/dev/catalog-import-gaps.md` (`mail-enable-enum — twenty` flipped to implemented), `docs/dev/capabilities.yml` (new id `mail-enable-enum`, version 1→2)
+
+Closes #302, extending [byo-outgoing-mail.md](byo-outgoing-mail.md) (#122). BYO mail injects malmo-canonical `MALMO_MAIL_*` and relies on the app's compose to adapt them, but compose `environment:` interpolation can only substitute / default (`:-`) / test-presence (`:+`) — it cannot **remap one enum's tokens onto another's**. Two shipped apps are degraded purely by this: twenty's `EMAIL_DRIVER` (`logger|smtp`, boot-rejects the empty string the `${MALMO_MAIL_HOST:+smtp}` gate expands to on an unbound box) and vaultwarden's `SMTP_SECURITY` (`off|starttls|force_tls`, a value remap off malmo's `none|starttls|tls`). This generalizes the projection the brain already hand-computes for `MALMO_MAIL_USE_TLS`/`USE_SSL`/`DSN` (which `mail.go` does in Go *because* "compose can't derive these from the string") into a manifest-declared value map.
+
+## What was done
+
+### Manifest (`internal/manifest/manifest.go`)
+
+- `Mail` gains `Env map[string]MailEnvMap`; new `MailEnvMap{From, Map}`. Each key is the app's own env-var name; `from` names a mail-state domain and `map` gives the app's token for each domain value.
+- Two domains: `encryption` (`none|starttls|tls`, the bound provider's mode — treated as `none` when unbound) and the synthetic `bound` (`bound|unbound`). Domain names/values are exported constants (`MailFromEncryption`, `MailFromBound`, `MailBound`, `MailUnbound`) so lifecycle resolution references them, not string literals. The `encryption` domain values are kept as literals here (not imported from `store`) so the manifest layer stays free of a `store` dependency; a lifecycle test guards the two vocabularies against drift.
+- `validateMail` extended: each env name is validated exactly like a config `app_env` (reuses `configEnvName` + the `MALMO_` / `reservedConfigEnv` guards — the token lands in `.env` under the app's own name, so it can't be allowed to clobber the `MALMO_` family or a loader var), `from` must be a known domain, and `map` must cover that domain **exactly** (right count + every key present + non-empty token) so resolution never hits an undeclared token.
+
+### Lifecycle (`internal/lifecycle/mail.go`, `lifecycle.go`)
+
+- New `mailAppEnvLines(mail, bound)` resolves each declared var to its token for the current state and renders `APP_VAR=token`, keys sorted for a byte-stable `.env`. `bound == nil` (unbound) resolves the `none`/`unbound` tokens. Validation guarantees the map covers the domain, so the lookup can't miss.
+- **Emitted in both states** — the load-bearing change. Unlike `MALMO_MAIL_*` (bound-only), the declared vars are stamped whether or not a provider is bound, so an enum-driver app is present-and-valid unbound instead of boot-rejecting on an empty value. `writeEnv` (install) and `rewriteEnvMail` (rebind) both take `man.Mail` (already in scope at each call site — no extra manifest read) and append `mailAppEnvLines` after the `MALMO_MAIL_*` block.
+- `rewriteEnvMail` now strips the declared vars **by name** (not just the `MALMO_MAIL_` prefix) before re-resolving, so a rebind — including an unbind, which drops `MALMO_MAIL_*` but keeps the enum vars with their unbound tokens — re-stamps cleanly with no duplicate lines.
+
+## Verification
+
+- `internal/manifest`: `mail.env` maps parse for both domains; `validateMail` rejects unknown `from`, an incomplete domain, an extra key, a misspelled key at the right count, an empty token, a `MALMO_`-prefixed name, a lowercase name, and a reserved runtime name (`PATH`).
+- `internal/lifecycle`: install unbound stamps `EMAIL_DRIVER=logger` + `SMTP_SECURITY=off` and no `MALMO_MAIL_*`; install bound stamps `smtp`/`force_tls` alongside the `MALMO_MAIL_*` family; rebind bound→unbound→bound re-stamps the right tokens each way with exactly one line per var (no duplicates); a unit test drives `mailAppEnvLines` across all three encryption modes (also guarding the manifest-literal ↔ `store.MailEncryption*` alignment) and asserts an empty `mail.env` yields no lines (existing mail-only apps unaffected).
+- Coverage on new code: `mailAppEnvLines` 100%, `validateMail` 100%; the mail branches added to `writeEnv`/`rewriteEnvMail` are all exercised (their residual uncovered lines are pre-existing `os` error paths).
+- `make check` green.
+
+## Known gaps & deviations
+
+- **Catalog/store manifest wiring is a coordinated follow-up, not in this repo.** The `catalog/` tree was cut over to the store repo (`07aac3a`), so `catalog/twenty` (`EMAIL_DRIVER`) and the store repo's `apps/vaultwarden` (`SMTP_SECURITY`, its `outbound-mail-enum-mismatch` limitation) — plus the issue's live send checks for both — ship separately against this brain mechanism. The ledger entry is flipped to `implemented` per the "mechanism shipped" convention and names both apps for the revisit.
+- **Reserved-name validation on `mail.env` keys is slightly beyond the issue's literal list** (it named only the domain-coverage + unknown-`from` rules). Kept deliberately: these vars land in the same `.env` as `MALMO_MAIL_*`, so a `MALMO_`-prefixed or loader-var key could clobber a brain-owned or process-critical value; the guard reuses the existing config `app_env` validator, matching the issue's "same direct-stamp convention as `config:`" framing.
+- **Compose still needs a pass-through line** (`EMAIL_DRIVER: ${EMAIL_DRIVER}`) in the app's compose — the remap happens in Go, the compose just forwards the resolved value. That line is part of the store-repo manifest work above.
+
+## What's next
+
+- Wire `catalog/twenty` (`EMAIL_DRIVER` via `from: bound`) and the store repo's `apps/vaultwarden` (`SMTP_SECURITY` via `from: encryption`), flip vaultwarden's `outbound-mail-enum-mismatch` and twenty's row toward `full` after a live send check.
+- The `capabilities.yml` id `mail-enable-enum` now fires the mechanical re-screen of any app that recorded a wait on that gap-class (`CAPABILITIES.md`).

docs/specs/APP_MANIFEST.md

@@ -256,6 +256,22 @@ environment:
   MAILER_FROM: "${MALMO_MAIL_FROM:-kimai@example.com}"
 ```
 
+**Enum-style config a compose gate can't reach (`mail.env`).** Some apps take a mail setting as a *boot-validated enum* rather than a boolean or free string — twenty's `EMAIL_DRIVER` (`logger` | `smtp`, rejects empty) or vaultwarden's `SMTP_SECURITY` (`off` | `starttls` | `force_tls`, whose tokens differ from malmo's `none`/`starttls`/`tls`). Compose `environment:` interpolation can only substitute, default (`:-`), or test-presence (`:+`); it cannot *remap* one enum's tokens onto another's, and the established `${MALMO_MAIL_HOST:+…}` gate expands to the empty string on an unbound box, which boot-rejects the enum. So the manifest declares the mapping and the brain resolves the final token in Go, stamping it into the `.env` under the app's own name — no `MALMO_` indirection, the same direct-stamp convention `config:` uses (# D4):
+
+```yaml
+mail:
+  optional: true
+  env:
+    EMAIL_DRIVER:                                    # twenty — is a provider bound?
+      from: bound                                    # domain: bound | unbound
+      map: { bound: smtp, unbound: logger }
+    SMTP_SECURITY:                                   # vaultwarden — the provider's mode
+      from: encryption                               # domain: none | starttls | tls (unbound ⇒ none)
+      map: { none: "off", starttls: starttls, tls: force_tls }
+```
+
+Each entry's `from` names a mail-state domain — `bound` (whether any provider is bound) or `encryption` (the bound provider's mode, treated as `none` when unbound) — and `map` must cover that domain exactly (every value, none extra), so resolution never hits an undeclared token. The var name is validated like a config `app_env` (uppercase identifier, never the `MALMO_` prefix or a loader/runtime var). **Unlike the rest of `MALMO_MAIL_*`, these vars are stamped in *both* states:** unbound resolves the `unbound`/`none` tokens, so the enum is always present and valid and the app never boot-rejects on an empty value. The app's compose passes the resolved value straight through (`EMAIL_DRIVER: ${EMAIL_DRIVER}`) — no remap, because the brain already did it.
+
 v1 admits only `optional: true` — an app that *can't* run unbound (`optional: false` or a bare `mail: {}`) is rejected at parse, because a box with no registered providers couldn't install it. Required-mail semantics (blocking install until a provider is picked) is a possible later loosening; declare-and-degrade is the v1 contract.
 
 ### D4. User-supplied configuration