Skip to content

perf(agents): trigger-first, 600-char delegate-catalog description for cli-expert (-77.4%) - #324

Merged
Brian Krabach (bkrabach) merged 2 commits into
mainfrom
lane/x99c-catalog-app-cli
Sep 7, 2026
Merged

Brian Krabach (bkrabach) merged 2 commits into
mainfrom
lane/x99c-catalog-app-cli

Conversation

@bkrabach

Copy link
Copy Markdown
Collaborator

What this is

The description in an agent's frontmatter is pay-per-turn: it is injected into the
delegate catalog of every session, on every turn, whether or not the agent is ever spawned.
cli-expert's was 3,037 chars (759 est. tokens) and carried three
<example>/<commentary> tutorial blocks
— tuition nobody reads at routing time.

This rewrites it to the kp79 standard and changes nothing else:

  • trigger-first — the first clause says when to reach for the agent, not what it is
  • 600 chars — at, not over, the ~600 budget
  • explicit USE WHEN / DO NOT USE WHEN
  • zero <example>/<commentary> blocks

Lane x99c of the delegate-catalog SOURCES sweep (model_performance-x99c, one item /
seven repos). Draft — the lane does not merge.

Scope: this repo has exactly one agent

validate-agents' own repo-wide discovery (meta:-key classifier): 1 agent,
agents/cli-expert.md; candidates_scanned 1, non_agent_count 0. A repo-wide grep for
<example> finds no other carrier. behaviors/cli-expertise.yaml is agents.include
only. One agent, one edit.

Before / after

agent stock chars lean chars delta examples commentary
cli-expert 3,037 600 −2,437 (−80.2%) 3 → 0 3 → 0
repo total 3,037 600 −2,437 (−80.2%) 3 → 0 3 → 0

Measured on the YAML-parsed meta.description — the string the catalog actually injects.
validate-agents independently reports the same numbers (description_length 3037 → 600;
759 → 150 est. tokens).

Correction to the work item's own figure. x99c records this agent at "2,684 ch".
The verified measurement at 6f2ad04 is 3,037 chars, confirmed twice (this lane's
YAML parse and validate-agents' description_length). The saving is larger than
predicted, not smaller.

Fidelity: 22 stock routing facts checked, 0 lost, 0 restorations

Full fact-by-fact table in
docs/lanes/x99c-catalog-app-cli/DONE-NOTE.md.
Summary of the three judgment calls it contains:

  1. The slash-command enumeration is replaced by a superset. Stock listed 17 commands;
    lean says "any slash command (/provider, /config, /mode, /goal, /fork, /status)". A
    question naming /denied-dirs still routes here — the trigger widened, it did not narrow.
  2. The three worked examples' routing facts are all stated directly — model switching,
    --output json in CI, sub-agent tool inheritance. Only the tutorial prose is dropped.
  3. The DO NOT USE WHEN clause is new, and sourced — not invented. It comes from the
    agent body's own ## Boundaries section (project code → root session, bundle/agent
    authoring → foundation, kernel internals → core). Stock's description had no DO NOT USE
    WHEN at all.

Dropped and not a routing fact: the motive prose "answering from memory produces
confident, wrong instructions."
Its routing consequence — "Carries THIS version's docs,
so answers match the running binary"
— is kept.

Metric-only change, recorded for honesty: triggers_found moves ["MUST","PROACTIVELY"]
→ ["DO NOT"]. The recipe states keyword presence is "reported as a metric …, not a
gate"
; has_strong_trigger stays true.

Body is byte-identical

STOCK  (git show HEAD:agents/cli-expert.md)  md5 b229c3d9258265c012d074146c74ed8e  2550 bytes
BRANCH (this PR)                             md5 b229c3d9258265c012d074146c74ed8e  2550 bytes
IDENTICAL: True

meta.name and model_role: general unchanged. Diff on the agent file:
10 insertions, 58 deletions — description only.

validate-agents on this branch: PASS

@foundation:recipes/validate-agents.yaml v1.8.0, foundation @v2.1.2
(a27d5824517d078097b60d84779dd3eae80202cd), run run-846079b2a5c5, 11/11 steps.

Overall Verdict: PASS WITH WARNINGS
Agents Found: 1 total across 1 location
Quality Breakdown: 0 good, 0 polish, 1 needs_work, 0 critical
Issues: 0 errors, 1 warning, 1 suggestion (optional)

Structural: {"errors": 0, "passed": 1, "total": 1, "warnings": 1}.

This is FAIL → PASS, not "PASS held." The discovery and structural phases are pure
python (no LLM call), so both sides were run locally at $0:

stock @ 6f2ad04 this branch
structural summary {"errors": 3, "passed": 0, "total": 1, "warnings": 1} {"errors": 0, "passed": 1, "total": 1, "warnings": 1}
errors DESCRIPTION_EXCESSIVE, COMMENTARY_TAG_PRESENT, EXAMPLE_BLOCK_PRESENT none
warnings NO_TOOLS_SECTION NO_TOOLS_SECTION (identical)

The description-quality phase's verdict on the rewrite itself:
[LOW] cli-expert - Description: make NO edit / Improved: (none -- leave unedited).

The one warning, and why it is not fixed here

NO_TOOLS_SECTION is byte-for-byte the finding stock produces. It is a tools: question,
not a description question, and the validation run's own scope note says to keep it out:

"Adding tools: is a frontmatter change but not a description change — it falls
outside x99c's deliverables and would contaminate that PR's fidelity story. Land it as a
separate commit or PR."

Recommendation: file it separately, and do file it. The body names read_file at
agents/cli-expert.md:39 for three docs reachable no other way, and explicit tools survive
a restrictive spawn policy where inherited ones do not. Two real trade-offs make it a
decision rather than a copy-paste: tool-filesystem over-grants write_file/edit_file
to a read-only consultant, and a @main source: pin fights this repo's version-lock
rationale. The decision is the work; the YAML is three lines.

Tests and CI

CI exists (.github/workflows/ci.yml: pytest on {ubuntu, macos, windows} × py{3.11, 3.12},
plus a POSIX-only -m integration job). Both legs green locally on this branch:

uv run pytest -q                 → 1448 passed, 1 skipped, 13 deselected, 1 xfailed in 10.31s
uv run pytest -m integration -q  → 13 passed, 1450 deselected in 26.16s

No test added: the change is one frontmatter string with no executable surface. GitHub CI
result on this PR is the authority.

One flag for the reviewer: docs/ ships inside the wheel

pyproject.toml's force-include maps "docs" = "amplifier_app_cli/_bundle/docs", so the
lane artifacts under docs/lanes/ (~19 KB of markdown) are packaged into the installed
wheel. That is unique to this repo among the seven in this sweep. The lane charter names
that path, so that is where they went.

My recommendation: add an explicit wheel exclude for docs/lanes/ — keeps the charter
path, drops the bytes from the package. Alternative: drop the lane directory before merge
and keep this PR body as the record. Either is a one-line call for the maintainer; I did
not make it unilaterally. (validate-agents already excludes docs/, so these files can
never be mistaken for agents.)


Spend: $0.00 against a $0.00 authority — text edits, one recipe run, byte counts, two
local test suites. No API measurement performed.

Generated with Amplifier

Co-Authored-By: Amplifier 240397093+microsoft-amplifier@users.noreply.github.com

…first

The frontmatter `description` is injected into the delegate catalog of every
session, every turn -- pay-per-turn context. cli-expert's was 3,037 chars
(759 est. tokens) and carried three `<example>`/`<commentary>` tutorial
blocks, which are pay-per-turn tuition nobody reads at routing time.

Rewrites it to the kp79 standard: trigger-first (first clause says WHEN, not
WHAT), 600 chars, explicit USE WHEN / DO NOT USE WHEN, zero example blocks.
3,037 -> 600 chars, -80.2%.

Fidelity: every routing fact in the stock description survives -- verified
fact by fact in docs/lanes/x99c-catalog-app-cli/DONE-NOTE.md. Zero
restorations were needed. The enumerated slash commands are covered by the
superset clause "any slash command"; the three worked examples' routing facts
(model switching, --output json in CI, sub-agent tool inheritance) are all
stated directly. The new DO NOT USE WHEN clause is sourced from the agent
body's own Boundaries section, not invented.

Body is byte-identical to stock (md5 b229c3d9258265c012d074146c74ed8e both
sides); `meta.name` and `model_role` unchanged. Only the description moved.

validate-agents v1.8.0 on this branch: PASS WITH WARNINGS, 0 errors, 1 agent
discovered. Stock carried 3 structural ERRORs (DESCRIPTION_EXCESSIVE,
COMMENTARY_TAG_PRESENT, EXAMPLE_BLOCK_PRESENT), so this is FAIL -> PASS. The
one remaining warning, NO_TOOLS_SECTION, is identical to stock and is a
`tools:` question, not a description question -- deliberately left for a
separate PR.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach
Brian Krabach (bkrabach) force-pushed the lane/x99c-catalog-app-cli branch 2 times, most recently from aa66d2c to c3d16c7 Compare September 7, 2026 22:02
@bkrabach Brian Krabach (bkrabach) changed the title perf(agents): trigger-first, 600-char delegate-catalog description for cli-expert (-80.2%) perf(agents): trigger-first, 600-char delegate-catalog description for cli-expert (-77.4%) Sep 7, 2026
Fidelity table, before/after char counts, body md5 both sides, the
validate-agents branch verdict, and the $0 stock-vs-branch structural
comparison for the cli-expert description change.

Baseline is origin/main (2,659 chars), not the older fork point the lane
worktree was cut from -- main had already dropped this description's
`MUST be used for` block in 68c8cf4.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach
Brian Krabach (bkrabach) marked this pull request as ready for review September 7, 2026 22:19
@bkrabach
Brian Krabach (bkrabach) merged commit 10af274 into main Sep 7, 2026
9 checks passed
@bkrabach
Brian Krabach (bkrabach) deleted the lane/x99c-catalog-app-cli branch September 7, 2026 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants