Skip to content

[usage] Stop a run at a cumulative token budget (per run and per session) #179

Closed

Description

@yusiwen

Context

Today the only limits on spending are a per-run step count (MaxSteps, 20/50/15/20 per agent,
enforced at agent/agent.go:190) and a per-request output cap (MaxTokens = 4096). Neither
accumulates: a 50-step run on a big history can spend an unbounded number of prompt tokens, and
nothing stops it partway.

Until #174 landed there was no input to build this on. It now exists: provider-reported usage reaches
Agent.UsageTotal (agent/agent.go) and the TUI through StreamCallbacks.OnUsage, so a bound can be
computed from what the provider actually charged rather than from len(text)/4.

Roadmap entry A3 (docs/roadmap.md:79-91), picked up from tracker #168. Its evidence line
(rg -in 'circuit' . -g '!*_test.go' → exit 1) is still true at the baseline it was written against,
and nothing reads UsageTotal to make a decision today.

What to build

  • S1 — the budget value and its config surface. config.BudgetConfig with
    max_tokens_per_run and max_tokens_per_session (both 0/absent = unlimited, which is today's
    behaviour), merged like every other section, shown in config.example.json, and wired onto the
    agent in main.go.
  • S2 — enforcement in the ReAct loop. The agent tracks the current run's spend separately from the
    session total it already keeps, and checks both before every provider call — the only point that
    spends. Crossing either budget ends the run there, with a message that names which budget fired, the
    limit and the spend. No provider call is made to produce that message: a summarize-then-stop would
    spend the budget it is enforcing. The loop's existing MaxSteps treatment (a limit reached returns a
    message, not an error) is the shape to follow.
  • S3 — visibility. When a limit is configured, the status bar reads tokens: spent/limit;
    with no limit configured it renders exactly as it does today, so the 27 committed frame goldens stay
    byte-identical.
  • S4 — tests and docs. The loop tests count provider calls (the property is when it stops, not
    only that it stops); docs/roadmap.md A3 carries its issue link; CHANGELOG.md and CODEBASE.md
    updated.

Design decisions taken here — say so if you disagree

  • Tokens, not money. Cost needs a per-model price table, which does not exist and which changes
    independently of this code. Tokens are what providers report today; a price table is its own
    decision (which models, which currency, refreshed how).
  • Off by default. Both limits default to unlimited, so no existing user's behaviour changes.
  • Stop before the call, not after a step. The loop checks at the top, so the run cannot overshoot
    by one call beyond the budget. An overshoot can still be reported (the call that crossed the line
    already happened), and the message says so with real numbers.
  • Reported usage, else an estimate. A provider that reports nothing must not silently disable the
    boundary; for such a call the spend falls back to the response estimate, and that fallback is
    documented rather than hidden.
  • Sub-agents have their own budgets. A task sub-agent builds its own agent, so its spend does not
    count against the parent's. Cross-agent attribution is roadmap G1; pretending otherwise would be a
    number that is not computable.

Acceptance

  • With a run budget of N and a mock provider reporting M tokens per call, the provider is called
    exactly ceil(N/M) times — that is the comparison: a counted call total, not a timing.
  • With no budget configured, a 10-step run makes exactly the same number of provider calls as before
    this change (a counted control, so "off by default" is asserted rather than claimed).
  • The run budget and the session budget are enforced independently: two runs on one agent, the first
    spending past the session limit, leave the second making zero provider calls and returning the
    session message.
  • The message names the budget that fired (run or session), the limit, and the spend.
  • A provider that reports no usage still trips a small budget, via the documented estimate fallback.
  • The status bar shows tokens: spent/limit when a limit is configured and exactly today's text when
    it is not; the 27 committed frame goldens are unchanged.
  • go test ./... -count=1 and -race green; make staticcheck clean.

Out of scope

  • Cost and pricing, for the reason above.
  • A breaker for a model that repeats an identical tool call — roadmap F3.
  • Attribution of a sub-agent's spend to its parent — roadmap G1.

Activity

  1. added
    featureA new capability rather than an improvement to existing behaviour
    on Oct 10, 2026
  2. yusiwen commented on Oct 10, 2026

    @yusiwen
    OwnerAuthor

    Landed as PR #180, rebase-merged to master as
    116301b.

    Delivered (the S1–S4 increments in the issue):

    • S1 — config.BudgetConfig (budget.max_tokens_per_run, budget.max_tokens_per_session) and
      Config.TokenBudgets(). Both default to 0 = unlimited; a negative value is ignored rather than read
      as "no limit".
    • S2 — BudgetTokensPerRun / BudgetTokensPerSession on the agent, a per-run runUsage reset by
      each Run, and the check at the top of the ReAct loop, before the call that would spend. Reaching
      either limit returns a message naming the budget, the spend and the knob to raise.
    • S3 — the status bar reads tokens: spent/limit when a session budget is configured and is
      byte-identical to before when none is.
    • S4 — tests, plus docs/roadmap.md, CHANGELOG.md, CODEBASE.md and config.example.json.

    Measurements

    • go test ./... -count=1 and -race: 13 packages ok, 0 failed, before and after the merge.
    • CI: nine checks green, 11 annotations = the baseline, per job. No new annotations.
    • Six mutations of the new behaviour each failed on an assertion; the comparison class is a counted
      provider-call total (ceil(N/M) calls for a budget of N at M tokens per call), with an exact-call
      count control for "off by default".
    • The 27 committed frame goldens pass unchanged, because the fraction appears only when a session
      budget is configured.

    One deliberate change to #174's contract, called out here rather than buried: a call whose endpoint
    reports nothing now advances Agent.UsageTotal by an estimate of its output, where before it added
    nothing. A budget that a silent endpoint could switch off is not a budget. The estimate never fires
    OnUsage, so no consumer is handed an estimate as a provider number, and #174's test was updated to
    assert both halves instead of being left green by accident.

    Not delivered (and why):

    • Cost and pricing: they need a per-model price table, which no command in this repository can
      produce. A3's title says "token or cost"; the cost half is now the only part of that entry still
      open.
    • Attribution of a sub-agent's spend to its parent: roadmap G1. Sub-agents enforce the same limits
      against their own spend, so delegation can still exceed a session budget in aggregate — a named gap
      rather than a number that cannot be computed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureA new capability rather than an improvement to existing behaviour

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions