feat(flow): expand codex control plane and daemon APIs

Author: nikivdev

docs/codex-first-control-plane-roadmap.md

@@ -63,12 +63,24 @@ The result is "good pieces, weak control plane".
 
 These are enough to start. The missing work is unification.
 
+## Constraint: no Codex fork
+
+Flow should target current upstream Codex directly.
+
+That means:
+
+- prefer wrapper transport + config over patching Codex
+- use stable upstream surfaces like normal user skill roots, `skills/list`, and `thread/*`
+- treat newer upstream features such as `skills/list perCwdExtraUserRoots` and in-process app-server clients as accelerators, not prerequisites
+- keep repo-specific behavior in Flow or repo executors, not in a private Codex fork
+
 ## Proposed Architecture
 
-### 1. `codexd`: long-lived Codex control daemon
+### 1. warm Codex control layer
 
-Add a Flow-managed daemon, either as an extension of `ai-taskd` or as a focused
-`codexd`, with one warm `codex app-server` connection per repo.
+Add a Flow-managed warm control layer, either as an extension of `ai-taskd`, a
+focused `codexd`, or a lighter in-process broker where that is enough for the
+current upstream Codex client surface.
 
 Responsibilities:
 

docs/commands/ai.md

@@ -127,7 +127,14 @@ Notes:
 
 Flow can also materialize tiny per-launch runtime skills for current upstream Codex without forking Codex.
 
-Enable it with:
+Enable it globally with:
+
+```bash
+f codex enable-global --full
+f codex doctor --path ~/docs --assert-runtime --assert-schedule
+```
+
+Or configure it manually with:
 
 ```toml
 [codex]
@@ -148,9 +155,25 @@ Inspect or clear runtime state:
 ```bash
 f codex runtime show
 f codex runtime clear
+f codex memory status
+f codex memory query --path ~/code/flow "codex control plane runtime skills"
+f codex memory recent --path ~/docs
 f codex doctor
 ```
 
+Assertive health checks:
+
+```bash
+f codex doctor --path ~/docs --assert-runtime
+f codex doctor --path ~/docs --assert-schedule
+f codex doctor --path ~/docs --assert-learning
+f codex doctor --path ~/docs --assert-autonomous
+```
+
+`--assert-learning` is intentionally strict: it fails until Flow has real
+logged events, grounded outcome samples, and a non-empty scorecard for that
+target.
+
 Built-in plan writer:
 
 ```bash
@@ -169,16 +192,32 @@ Codex usage history without replaying Codex in the hot path.
 Useful commands:
 
 ```bash
+f codex memory sync --limit 400
+f codex memory recent --path ~/work/example-project --limit 12
 f codex skill-eval show --path ~/work/example-project
 f codex skill-eval run --path ~/work/example-project
 f codex skill-eval cron --limit 400 --max-targets 12 --within-hours 168
 f codex skill-source list --path ~/work/example-project
 f codex skill-source sync --path ~/work/example-project --skill find-skills
 ```
 
+The Codex memory mirror:
+
+- stores durable indexed memory under the Jazz2 root (`~/.jazz2/...` or `~/repos/garden-co/jazz2/.jazz2/...`)
+- mirrors Flow’s route/outcome history into SQLite with WAL enabled
+- extracts compact repo/code facts from repo capsules (summary, commands, important paths, docs hints)
+- adds bounded live code-path retrieval for explicit repo references, so prompts like `see ~/code/flow ...` can inject likely files such as `src/ai.rs` or `docs/...` without dumping raw source
+- indexes durable repo entrypoints and extracted symbols under the same Jazz2-rooted memory store, then supplements them with live symbol extraction from the top-ranked code files during `memory query` / `codex resolve`
+- adds tiny symbol snippets for the top code hits, so coding prompts can carry actual struct/function shape without inlining whole files
+- biases retrieval by intent: implementation/file-edit prompts prefer symbols, snippets, and `src/...` paths; summary/docs prompts prefer doc headings and docs paths
+- stays best-effort so failed memory writes do not block normal Codex coding turns
+- is refreshed again by `f codex skill-eval cron`, so the mirror heals even if a hot-path write is skipped
+- is queried automatically for explicit repo references during `f codex open` / `f codex resolve`
+
 What `cron` does:
 
 - scans only recent logged Flow Codex events
+- syncs recent skill-eval logs into the Jazz2-backed memory mirror
 - skips missing/moved repo paths
 - rebuilds scorecards for a bounded number of recent repos
 - never launches Codex or replays network work in the background
@@ -195,6 +234,8 @@ f codex-skill-eval-launchd-status
 f codex-skill-eval-launchd-logs
 ```
 
+`f codex enable-global --full` installs this schedule for you.
+
 Default schedule:
 
 - every 30 minutes

docs/commands/db.md

@@ -16,6 +16,17 @@ Create Jazz2 app credentials and populate env vars for the current project.
 f db jazz new --kind mirror --name gitedit-mirror
 ```
 
+Resolution order for the credential bootstrap helper:
+
+1. Local `jazz-tools` binary on `PATH`
+2. Pinned `npx --yes jazz-tools@0.20.14`
+
+For deliberate experiments, override the pinned fallback with:
+
+```bash
+FLOW_JAZZ_TOOLS_PACKAGE_SPEC=jazz-tools@<version> f db jazz new --kind mirror
+```
+
 ## Postgres
 
 Run Drizzle migrations for the default Postgres project (`~/org/la/la/server`).

docs/everruns-maple-runbook.md

@@ -48,8 +48,8 @@ For optimized mirror (remote ClickHouse + durable local spool), also set:
 
 ```bash
 f env set SEQ_CH_MODE=mirror
-f env set SEQ_CH_MEM_PATH=~/repos/ClickHouse/ClickHouse/user_files/seq_mem.jsonl
-f env set SEQ_CH_LOG_PATH=~/repos/ClickHouse/ClickHouse/user_files/seq_trace.jsonl
+f env set SEQ_CH_MEM_PATH=~/.config/flow/rl/seq_mem.jsonl
+f env set SEQ_CH_LOG_PATH=~/.config/flow/rl/seq_trace.jsonl
 ```
 
 ## 2) Run with env injected

docs/flow-toml-spec.md

@@ -146,6 +146,7 @@ fr = "f run"
   - `runtime_skills`: when true, `f codex open` may materialize Flow-managed per-launch runtime skills for wrapper transports.
   - `[[codex.reference_resolver]]`: repo-specific reference unrollers with wildcard `match` patterns and a shell `command` template.
   - command templates support `{{ref}}`, `{{query}}`, and `{{cwd}}`.
+  - `f codex enable-global --full` writes the global wrapper/runtime baseline into `~/.config/flow/flow.toml` for you.
 - `[skills.seq]`: optional defaults for `f skills fetch ...` (local seq scraper integration).
 - `[commit]`: optional commit workflow defaults; plain `f commit` uses fast commit + deferred Codex deep review by default. Set `quick-default = false` to make plain `f commit` run blocking review instead.
 - `[task_resolution]`: optional policy for nested task discovery (`f <scope>:<task>`, preferred scopes, and per-task routes when plain names collide).

docs/rl-signal-capture-runbook.md

@@ -27,7 +27,7 @@ From `~/code/flow`:
 export FLOW_RL_SIGNALS=true
 export FLOW_RL_SIGNALS_PATH=out/logs/flow_rl_signals.jsonl
 export FLOW_RL_SIGNALS_SEQ_MIRROR=true
-export FLOW_RL_SIGNALS_SEQ_PATH=~/repos/ClickHouse/ClickHouse/user_files/seq_mem.jsonl
+export FLOW_RL_SIGNALS_SEQ_PATH=~/.config/flow/rl/seq_mem.jsonl
 export FLOW_RL_SIGNAL_TEXT=snippet
 export FLOW_RL_SIGNAL_MAX_CHARS=4000
 ```

flow.toml

@@ -120,7 +120,7 @@ description = "Summarize RL signal counts, errors, and latency percentiles."
 
 [[tasks]]
 name = "rl-dataset-build"
-command = "python3 ./scripts/build_rl_runtime_dataset.py --harbor-dir ${HARBOR_DIR:-$HOME/repos/laude-institute/harbor} --flow-signals ${FLOW_RL_SIGNALS_PATH:-out/logs/flow_rl_signals.jsonl} --seq-mem ${SEQ_CH_MEM_PATH:-$HOME/repos/ClickHouse/ClickHouse/user_files/seq_mem.jsonl} --write-latest $@"
+command = "python3 ./scripts/build_rl_runtime_dataset.py --harbor-dir ${HARBOR_DIR:-$HOME/repos/laude-institute/harbor} --flow-signals ${FLOW_RL_SIGNALS_PATH:-out/logs/flow_rl_signals.jsonl} --seq-mem ${SEQ_CH_MEM_PATH:-$HOME/.config/flow/rl/seq_mem.jsonl} --write-latest $@"
 description = "Build Harbor-ready RL dataset snapshot from flow+seq runtime signals."
 
 [[tasks]]
@@ -491,12 +491,12 @@ variables = [
   { key = "FLOW_RL_SIGNALS", default = "true" },
   { key = "FLOW_RL_SIGNALS_PATH", default = "out/logs/flow_rl_signals.jsonl" },
   { key = "FLOW_RL_SIGNALS_SEQ_MIRROR", default = "true" },
-  { key = "FLOW_RL_SIGNALS_SEQ_PATH", default = "~/repos/ClickHouse/ClickHouse/user_files/seq_mem.jsonl" },
+  { key = "FLOW_RL_SIGNALS_SEQ_PATH", default = "~/.config/flow/rl/seq_mem.jsonl" },
   { key = "FLOW_ROUTER_SUGGESTED_TASK", default = "" },
   { key = "FLOW_ROUTER_OVERRIDE_REASON", default = "" },
   { key = "FLOW_RL_SIGNAL_TEXT", default = "snippet" },
   { key = "FLOW_RL_SIGNAL_MAX_CHARS", default = "4000" },
-  { key = "SEQ_CH_MEM_PATH", default = "~/repos/ClickHouse/ClickHouse/user_files/seq_mem.jsonl" },
+  { key = "SEQ_CH_MEM_PATH", default = "~/.config/flow/rl/seq_mem.jsonl" },
   { key = "HARBOR_DIR", default = "~/repos/laude-institute/harbor" },
 ]
 

scripts/build_rl_runtime_dataset.py

@@ -456,7 +456,7 @@ def main() -> int:
     parser = argparse.ArgumentParser(description="Build RL runtime dataset from flow + seq logs")
     parser.add_argument("--harbor-dir", default=str(Path("~/repos/laude-institute/harbor").expanduser()))
     parser.add_argument("--flow-signals", default="out/logs/flow_rl_signals.jsonl")
-    parser.add_argument("--seq-mem", default=str(Path("~/repos/ClickHouse/ClickHouse/user_files/seq_mem.jsonl").expanduser()))
+    parser.add_argument("--seq-mem", default=str(Path("~/.config/flow/rl/seq_mem.jsonl").expanduser()))
     parser.add_argument("--snapshot", default="", help="snapshot name; default timestamp")
     parser.add_argument("--flow-last", type=int, default=20_000)
     parser.add_argument("--seq-last", type=int, default=50_000)