diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index dd918ff0..26b4f2e0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -50,5 +50,16 @@ jobs: - name: Build run: npm run build + # The unit suite (713 tests) was never wired in here — only the lifecycle + # integration test ran, so a failing unit test could merge unnoticed. + - name: Unit tests + run: npm test + + # Fails when a number in the docs disagrees with brand-numbers.json. + # Offline by design: it reads the committed snapshot and never fetches, so + # a blockrun.ai deploy in progress cannot fail this repo's CI. + - name: Brand numbers + run: node scripts/sync-brand-numbers.mjs --check + - name: Integration tests (lifecycle) run: npx vitest run --config vitest.integration.config.ts test/integration/lifecycle.test.ts diff --git a/.prettierignore b/.prettierignore index 1a993212..a5707f9e 100644 --- a/.prettierignore +++ b/.prettierignore @@ -1,3 +1,8 @@ dist/ node_modules/ package-lock.json + +# Vendored byte-for-byte from BlockRunAI/blockrun:brand/sync-brand-numbers.mjs. +# Formatting it here would fork the copy from the source and from the other 36 +# repos that carry it, and CI compares them. +scripts/sync-brand-numbers.mjs diff --git a/CLAUDE.md b/CLAUDE.md index 96c1dcb1..e5b57b8e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,6 @@ # ClawRouter -Smart LLM router for autonomous agents. 55+ models. Wallet-based auth. USDC micropayments via x402. +Smart LLM router for autonomous agents. 66 models. Wallet-based auth. USDC micropayments via x402. ## Commands @@ -56,4 +56,4 @@ src/ - Node >= 22 - MIT license - npm registry: `@blockrun/clawrouter` -- 15-dimension scoring for model routing (all local, < 1ms) +- 15-dimension scoring for model routing (all local, < 1ms) diff --git a/README.md b/README.md index d9746b60..8dd5cf80 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@

Agents can't sign up for accounts. Agents can't enter credit cards.
Agents can only sign transactions.

ClawRouter is the only LLM router that lets agents operate independently.

-8 models free, no crypto required. No signup. No API key. No credit card.

+8 models free, no crypto required. No signup. No API key. No credit card.


@@ -34,7 +34,7 @@ Agents can only sign transactions.

-> **ClawRouter** is an open-source smart LLM router that reduces AI API costs by up to 92%. It analyzes each request across 15 dimensions and routes to the cheapest capable model in under 1ms, entirely locally. ClawRouter is the only LLM router built for autonomous AI agents — it uses wallet signatures for authentication (no API keys) and USDC micropayments via the x402 protocol (no credit cards). 55+ models from OpenAI, Anthropic, Google, xAI, DeepSeek, and more. MIT licensed. +> **ClawRouter** is an open-source smart LLM router that reduces AI API costs by up to 87%. It analyzes each request across 15 dimensions and routes to the cheapest capable model in under 1ms, entirely locally. ClawRouter is the only LLM router built for autonomous AI agents — it uses wallet signatures for authentication (no API keys) and USDC micropayments via the x402 protocol (no credit cards). 66 models from OpenAI, Anthropic, Google, xAI, DeepSeek, and more. MIT licensed. --- @@ -46,10 +46,10 @@ Every other LLM router was built for **human developers** — create an account, ClawRouter is built for the agent-first world: -- **Starts at $0** — 8 NVIDIA models are free forever (incl. 675B Mistral Large 3, 262K-context Qwen3-Next 80B + a vision-capable Nemotron Omni) +- **Starts at $0** — 8 NVIDIA models are free forever (incl. 675B Mistral Large 3, 262K-context Qwen3-Next 80B + a vision-capable Nemotron Omni) - **No accounts** — a wallet is generated locally, no signup - **No API keys** — your wallet signature IS authentication -- **No model selection** — 15-dimension scoring picks the right model automatically +- **No model selection** — 15-dimension scoring picks the right model automatically - **No credit cards** — agents pay per-request with USDC via [x402](https://x402.org) - **No trust required** — runs locally, <1ms routing, zero external dependencies @@ -59,16 +59,16 @@ This is the stack that lets agents operate autonomously: **x402 + USDC + local r ## How it compares -| | OpenRouter | LiteLLM | Martian | Portkey | **ClawRouter** | -| ---------------- | ----------------- | ---------------- | ----------------- | ----------------- | ----------------------- | -| **Models** | 200+ | 100+ | Smart routing | Gateway | **55+** | -| **Free tier** | Rate-limited | BYO keys | No | No | **8 models, no signup** | -| **Routing** | Manual selection | Manual selection | Smart (closed) | Observability | **Smart (open source)** | -| **Auth** | Account + API key | Your API keys | Account + API key | Account + API key | **Wallet signature** | -| **Payment** | Credit card | BYO keys | Credit card | $49-499/mo | **USDC per-request** | -| **Runs locally** | No | Yes | No | No | **Yes** | -| **Open source** | No | Yes | No | Partial | **Yes** | -| **Agent-ready** | No | No | No | No | **Yes** | +| | OpenRouter | LiteLLM | Martian | Portkey | **ClawRouter** | +| ---------------- | ----------------- | ---------------- | ----------------- | ----------------- | ---------------------------------------------------------------------- | +| **Models** | 200+ | 100+ | Smart routing | Gateway | **66** | +| **Free tier** | Rate-limited | BYO keys | No | No | **8 models, no signup** | +| **Routing** | Manual selection | Manual selection | Smart (closed) | Observability | **Smart (open source)** | +| **Auth** | Account + API key | Your API keys | Account + API key | Account + API key | **Wallet signature** | +| **Payment** | Credit card | BYO keys | Credit card | $49-499/mo | **USDC per-request** | +| **Runs locally** | No | Yes | No | No | **Yes** | +| **Open source** | No | Yes | No | Partial | **Yes** | +| **Agent-ready** | No | No | No | No | **Yes** | ✓ Open source · ✓ Smart routing · ✓ Runs locally · ✓ Crypto native · ✓ Agent ready @@ -78,7 +78,7 @@ This is the stack that lets agents operate autonomously: **x402 + USDC + local r ## Quick Start -> **No wallet? 8 models work free out of the box.** Install, run, and pin `free/mistral-large-3-675b` (or any of the 8) — no crypto, no signup, no balance required. Add USDC later when you want paid models. +> **No wallet? 8 models work free out of the box.** Install, run, and pin `free/mistral-large-3-675b` (or any of the 8) — no crypto, no signup, no balance required. Add USDC later when you want paid models. ### Option A — OpenClaw Agent @@ -107,7 +107,7 @@ openclaw gateway restart > **Using Claude Code?** Check out [BRCC](https://blockrun.ai/brcc.md) — it's purpose-built for Claude Code with the same smart routing and x402 payments. > -> **Using NousResearch Hermes?** See [ClawRouter-Hermes](https://github.com/BlockRunAI/ClawRouter-Hermes) — a Python plugin that wires Hermes into the ClawRouter proxy. Same wallet, same 55+ models, same x402 USDC settlement on Base & Solana. +> **Using NousResearch Hermes?** See [ClawRouter-Hermes](https://github.com/BlockRunAI/ClawRouter-Hermes) — a Python plugin that wires Hermes into the ClawRouter proxy. Same wallet, same 66 models, same x402 USDC settlement on Base & Solana. No OpenClaw required. ClawRouter runs as a local proxy on port 8402. @@ -118,7 +118,7 @@ npx @blockrun/clawrouter ``` **2. Fund your wallet** — optional, skip for free tier -Your wallet address is printed on first run. For paid models, send a few USDC on Base or Solana — $5 covers thousands of requests. To stay at $0, pin any of the 8 free models (e.g. `free/mistral-large-3-675b`) or use `/model free` inside OpenClaw. +Your wallet address is printed on first run. For paid models, send a few USDC on Base or Solana — $5 covers thousands of requests. To stay at $0, pin any of the 8 free models (e.g. `free/mistral-large-3-675b`) or use `/model free` inside OpenClaw. **3. Point your client at `http://localhost:8402`** @@ -188,12 +188,12 @@ response = client.chat.completions.create(model="blockrun/auto", messages=[...]) Choose your routing strategy with `/model `: -| Profile | Strategy | Savings | Best For | -| ---------------- | ------------------ | -------- | -------------------- | -| `/model free` | Free NVIDIA models | **100%** | $0 balance, learning | -| `/model auto` | Balanced (default) | 74-100% | General use | -| `/model eco` | Cheapest possible | 95-100% | Maximum savings | -| `/model premium` | Best quality | 0% | Mission-critical | +| Profile | Strategy | Savings | Best For | +| ---------------- | ------------------ | ---------------------------------------------------------------------------------- | -------------------- | +| `/model free` | Free NVIDIA models | **100%** | $0 balance, learning | +| `/model auto` | Balanced (default) | **87%** | General use | +| `/model eco` | Cheapest possible | **98%** | Maximum savings | +| `/model premium` | Best quality | 0% | Mission-critical | **Shortcuts:** `/model grok`, `/model br-sonnet`, `/model gpt5`, `/model o3` @@ -204,7 +204,7 @@ Choose your routing strategy with `/model `: **100% local routing. <1ms latency. Zero external API calls.** ``` -Request → Weighted Scorer (15 dimensions) → Tier → Best Model → Response +Request → Weighted Scorer (14 dimensions) → Tier → Best Model → Response ``` | Tier | ECO Model | AUTO Model | PREMIUM Model | @@ -214,7 +214,14 @@ Request → Weighted Scorer (15 dimensions) → Tier → Best Model → Response | COMPLEX | gemini-3.1-flash-lite ($0.25/$1.50) | gemini-3.1-pro ($2/$12) | claude-fable-5 ($10/$50) | | REASONING | deepseek-reasoner ($0.20/$0.40) | deepseek-reasoner ($0.20/$0.40) | claude-sonnet-4.6 ($3/$15) | -**Blended average: $2.05/M** vs $25/M for Claude Opus = **92% savings** +**87% cheaper than pinning Claude Opus 5** for the same traffic, on `auto`; **98%** on `eco`. + +Not an "up to" figure. The baseline, the workload mix and the token ratio are +published in [`savings-mix.json`](https://github.com/BlockRunAI/blockrun/blob/main/src/brand/savings-mix.json), +priced against the live catalog, so anyone can recompute it and get the same +answer. Models withheld from `/v1/models` are excluded from the mix — pricing a +public claim on a model you cannot look up is not defensible — which makes the +number conservative. --- @@ -382,7 +389,7 @@ No Surf account, no API key — settles directly to Surf's Base treasury in USDC ## Models & Pricing -55+ models across 9 providers, one wallet. **8 models are $0 — paid models start at fractions of a cent.** +66 models across 9 providers, one wallet. **8 models are $0 — paid models start at fractions of a cent.** > **💡 "Cost per request"** = estimated cost for a typical chat message (~500 input + 500 output tokens). Paid requests also carry a flat **$0.002/tx settlement fee** (covers on-chain gas; already included in the price the gateway quotes). Free models never pay it. @@ -637,7 +644,7 @@ npm test **The LLM router built for autonomous agents** -You're here. 55+ models, local smart routing, x402 USDC payments — the only stack that lets agents operate independently. +You're here. 66 models, local smart routing, x402 USDC payments — the only stack that lets agents operate independently. `curl -fsSL https://blockrun.ai/ClawRouter-update | bash` @@ -661,7 +668,7 @@ Run Claude Code with 50+ models, no rate limits, no Anthropic account, no phone **ClawRouter for NousResearch Hermes** -Python plugin that wraps the ClawRouter proxy for `hermes-agent`. Same 55+ models, same x402 USDC payments on Base & Solana, native Hermes ergonomics. +Python plugin that wraps the ClawRouter proxy for `hermes-agent`. Same 66 models, same x402 USDC payments on Base & Solana, native Hermes ergonomics. `pip install hermes-plugin-clawrouter` @@ -678,15 +685,15 @@ Python plugin that wraps the ClawRouter proxy for `hermes-agent`. Same 55+ model ## More Resources -| Resource | Description | -| ------------------------------------------------------ | ------------------------ | -| [Documentation](https://blockrun.ai/docs) | Full docs | -| [Model Pricing](https://blockrun.ai/models) | All models & prices | -| [Image Generation & Editing](docs/image-generation.md) | API examples, 8 models | -| [Routing Profiles](docs/routing-profiles.md) | ECO/AUTO/PREMIUM details | -| [Architecture](docs/architecture.md) | Technical deep dive | -| [Configuration](docs/configuration.md) | Environment variables | -| [Troubleshooting](docs/troubleshooting.md) | Common issues | +| Resource | Description | +| ------------------------------------------------------ | ----------------------------------------------------------------------- | +| [Documentation](https://blockrun.ai/docs) | Full docs | +| [Model Pricing](https://blockrun.ai/models) | All models & prices | +| [Image Generation & Editing](docs/image-generation.md) | API examples, 8 models | +| [Routing Profiles](docs/routing-profiles.md) | ECO/AUTO/PREMIUM details | +| [Architecture](docs/architecture.md) | Technical deep dive | +| [Configuration](docs/configuration.md) | Environment variables | +| [Troubleshooting](docs/troubleshooting.md) | Common issues | ### Blog @@ -705,11 +712,11 @@ Python plugin that wraps the ClawRouter proxy for `hermes-agent`. Same 55+ model ### What is ClawRouter? -ClawRouter is an open-source (MIT licensed) smart LLM router built for autonomous AI agents. It analyzes each request across 15 dimensions and routes to the cheapest capable model in under 1ms, entirely locally — no external API calls needed for routing decisions. +ClawRouter is an open-source (MIT licensed) smart LLM router built for autonomous AI agents. It analyzes each request across 15 dimensions and routes to the cheapest capable model in under 1ms, entirely locally — no external API calls needed for routing decisions. ### How much can ClawRouter save on LLM costs? -ClawRouter's blended average cost is $2.05 per million tokens compared to $25/M for Claude Opus, representing 92% savings. Actual savings depend on your workload — simple queries are routed to free models ($0/request), while complex tasks get premium models. +On the `auto` profile ClawRouter costs 87% less than pinning Claude Opus 5 for every request, and 98% less on `eco`. That is computed from a published workload mix rather than estimated — see [savings-mix.json](https://github.com/BlockRunAI/blockrun/blob/main/src/brand/savings-mix.json) for the baseline and assumptions. Actual savings depend on your workload — simple queries are routed to free models ($0/request), while complex tasks get premium models. ### How does ClawRouter compare to OpenRouter? diff --git a/brand-numbers.json b/brand-numbers.json new file mode 100644 index 00000000..312505d8 --- /dev/null +++ b/brand-numbers.json @@ -0,0 +1,34 @@ +{ + "$schema": "https://blockrun.ai/brand/numbers.schema.json", + "version": 1, + "models": { + "chatVisible": 66, + "totalVisible": 86, + "free": 8, + "freeWithheld": 17, + "image": 8, + "video": 5, + "music": 1, + "speech": 5, + "soundfx": 1, + "withFallback": 44, + "withFallbackAllEntries": 75 + }, + "clawrouter": { + "dimensions": 15, + "tiers": 4, + "profiles": 4, + "aliases": 202 + }, + "mcp": { + "tools": 19 + }, + "chains": { + "rpc": 40 + }, + "savings": { + "baselineModel": "anthropic/claude-opus-5", + "ecoVsBaselinePct": 98, + "autoVsBaselinePct": 87 + } +} diff --git a/package.json b/package.json index 51fb7ca3..794238b6 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "@blockrun/clawrouter", "version": "0.12.235", - "description": "Smart LLM router — save 85% on inference costs. 55+ models (8 free), one wallet, x402 micropayments.", + "description": "Smart LLM router — save 87% on inference costs. 66 models (8 free), one wallet, x402 micropayments.", "type": "module", "main": "dist/index.js", "types": "dist/index.d.ts", diff --git a/scripts/sync-brand-numbers.mjs b/scripts/sync-brand-numbers.mjs new file mode 100644 index 00000000..4feb5ced --- /dev/null +++ b/scripts/sync-brand-numbers.mjs @@ -0,0 +1,256 @@ +#!/usr/bin/env node +/** + * Sync marketing numbers from BlockRun's canonical brand artifact. + * + * This file is copied byte-for-byte into every public repo as + * scripts/sync-brand-numbers.mjs. It is a copy rather than an npm package on + * purpose: a package would mean 37 dependency bumps, and several consuming + * repos have no package.json at all. Zero dependencies, plain Node. + * + * node scripts/sync-brand-numbers.mjs rewrite markers in place + * node scripts/sync-brand-numbers.mjs --check exit 1 on drift, write nothing + * node scripts/sync-brand-numbers.mjs --refresh re-fetch the artifact first + * + * --check NEVER touches the network. PR CI must be deterministic and offline: + * if it fetched, a deploy in progress would fail every repo in the org at once. + * Freshness is the fan-out job's problem, not the pull request's. + * + * Markers look like: 66 + * and wrap the WHOLE token, so a badge URL, its alt text and the prose number + * can all regenerate from one key. + */ +import { readFileSync, writeFileSync, readdirSync, statSync } from "node:fs"; +import { join, relative, extname } from "node:path"; + +const ROOT = process.cwd(); +const SNAPSHOT = join(ROOT, "brand-numbers.json"); +// ORIGIN is tried first because it IS the truth — the mirror can only ever be +// as fresh as the last time someone refreshed it. The mirror exists so a repo +// can still sync while blockrun.ai is down, not to front the origin. +// +// The mirror is awesome-blockrun's own brand-numbers.json: that repo consumes +// the artifact like every other, and its snapshot doubles as the org's copy. +// One file, one role per repo, nothing to keep in step by hand. +const ORIGIN = "https://blockrun.ai/brand/numbers.json"; +const MIRROR = + "https://raw.githubusercontent.com/BlockRunAI/awesome-blockrun/main/brand-numbers.json"; + +const argv = new Set(process.argv.slice(2)); +const check = argv.has("--check"); +const refresh = argv.has("--refresh"); + +const SKIP_DIRS = new Set([ + "node_modules", ".git", "dist", "build", "out", ".next", "coverage", + "vendor", "target", "__pycache__", ".venv", "venv", +]); +const TEXT_EXT = new Set([".md", ".mdx"]); + +/* ── 1. numbers ──────────────────────────────────────────────────────────── */ + +async function loadNumbers() { + if (!refresh) { + try { + return JSON.parse(readFileSync(SNAPSHOT, "utf8")); + } catch { + fail( + `no brand-numbers.json in ${ROOT}\n` + + ` run with --refresh once to seed it from ${ORIGIN}`, + ); + } + } + for (const url of [ORIGIN, MIRROR]) { + try { + const res = await fetch(url, { signal: AbortSignal.timeout(10_000) }); + if (!res.ok) continue; + const json = await res.json(); + writeFileSync(SNAPSHOT, `${JSON.stringify(json, null, 2)}\n`); + return json; + } catch { + /* try the next source */ + } + } + fail(`could not refresh from ${MIRROR} or ${ORIGIN}`); +} + +/** Flatten nested numbers into dotted keys, ignoring $comment / rationale prose. */ +function flatten(obj, prefix = "") { + return Object.entries(obj).flatMap(([k, v]) => { + if (k.startsWith("$")) return []; + const key = `${prefix}${k}`; + if (v && typeof v === "object" && !Array.isArray(v)) return flatten(v, `${key}.`); + if (v === null) return []; + return [[key, v]]; + }); +} + +/* ── 2. renderers ────────────────────────────────────────────────────────── */ + +/** + * How a key becomes text. Default is the bare value. + * + * A marker may carry an `@modifier` — `` — which + * selects a renderer without changing which number is looked up. The modifier + * is what makes a key reusable: the same mcp.tools appears as a shields badge + * at the top of a README and as a bare "19 tools" in a table two screens down, + * and one marker still keeps the badge URL, its alt text and the label in step. + * + * Renderers are registered under the FULL marker name so a badge's label is + * written out rather than guessed from the key. + */ +const badge = (label) => (n) => + `${n} ${label}`; + +const RENDER = { + "mcp.tools@badge": badge("tools"), + "models.totalVisible@badge": badge("models"), + "models.chatVisible@badge": badge("models"), +}; +const render = (marker, value) => (RENDER[marker] ?? String)(value); + +/** `mcp.tools@badge` looks up `mcp.tools`. Unmodified markers are unaffected. */ +const keyOf = (marker) => marker.split("@")[0]; + +/* ── 3. marker rewriting ─────────────────────────────────────────────────── */ + +const esc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); +const OPEN_ANY = //g; +const CLOSE_ANY = //g; + +/** Byte ranges of fenced code blocks — markers inside them are documentation. */ +function fencedRanges(text) { + const ranges = []; + const fence = /^(\s*)(`{3,}|~{3,})[^\n]*$/gm; + let open = null; + for (let m; (m = fence.exec(text)); ) { + if (open === null) open = m.index; + else { + ranges.push([open, m.index + m[0].length]); + open = null; + } + } + return ranges; +} + +function syncFile(file, numbers, problems) { + const before = readFileSync(file, "utf8"); + const rel = relative(ROOT, file); + const fenced = fencedRanges(before); + const inFence = (i) => fenced.some(([a, b]) => i >= a && i < b); + const known = new Map(numbers); + const used = new Set(); + + // Markers actually present, so a file is only ever rewritten for what it uses + // and an @modifier is carried through to the renderer verbatim. + const markers = new Set(); + // A marker naming a key that does not exist is an error, never a silent + // no-op: a typo'd marker would otherwise sit there looking synced forever. + for (const [re, shown] of [ + [OPEN_ANY, (n) => ``], + [CLOSE_ANY, (n) => ``], + ]) { + for (const m of before.matchAll(re)) { + if (inFence(m.index)) continue; + markers.add(m[1]); + if (!known.has(keyOf(m[1]))) problems.push(`${rel}: unknown key ${shown(m[1])}`); + } + } + + let after = before; + for (const marker of markers) { + const key = keyOf(marker); + if (!known.has(key)) continue; + const value = known.get(key); + const pair = new RegExp( + `()([\\s\\S]*?)()`, + "g", + ); + after = after.replace(pair, (whole, open, inner, close, offset) => { + if (inFence(offset)) return whole; + // Nesting means the closing tag of an inner marker would be consumed by + // the outer one. Refuse rather than produce mangled output. + if (/`, "g"))] + .filter((m) => !inFence(m.index)).length; + const closes = [...before.matchAll(new RegExp(``, "g"))] + .filter((m) => !inFence(m.index)).length; + if (opens !== closes) problems.push(`${rel}: unbalanced marker br:${marker} (${opens} open, ${closes} close)`); + } + + return { before, after, changed: before !== after, used }; +} + +/* ── 4. walk ─────────────────────────────────────────────────────────────── */ + +function* walk(dir) { + for (const name of readdirSync(dir)) { + if (SKIP_DIRS.has(name)) continue; + const p = join(dir, name); + const s = statSync(p); + if (s.isDirectory()) yield* walk(p); + else if (TEXT_EXT.has(extname(name))) yield p; + } +} + +function fail(msg) { + console.error(`brand-numbers: ${msg}`); + process.exit(1); +} + +/* ── 5. run ──────────────────────────────────────────────────────────────── */ + +const raw = await loadNumbers(); +const numbers = flatten(raw); +const problems = []; +const drifted = []; +const everUsed = new Set(); + +for (const file of walk(ROOT)) { + const { before, after, changed, used } = syncFile(file, numbers, problems); + used.forEach((k) => everUsed.add(k)); + if (!changed) continue; + drifted.push({ file: relative(ROOT, file), before, after }); + if (!check) writeFileSync(file, after); +} + +if (problems.length) { + for (const p of problems) console.error(` ${p}`); + fail(`${problems.length} marker problem(s)`); +} + +if (check) { + if (drifted.length === 0) { + console.log(`brand-numbers: up to date (${everUsed.size} keys in use)`); + process.exit(0); + } + console.error("brand-numbers: these files disagree with brand-numbers.json\n"); + for (const { file, before, after } of drifted) { + const b = before.split("\n"); + const a = after.split("\n"); + for (let i = 0; i < Math.max(b.length, a.length); i++) { + if (b[i] !== a[i]) { + console.error(` ${file}:${i + 1}`); + console.error(` - ${(b[i] ?? "").trim()}`); + console.error(` + ${(a[i] ?? "").trim()}`); + } + } + } + console.error( + "\n fix with: node scripts/sync-brand-numbers.mjs && git commit -am 'chore: sync brand numbers'", + ); + process.exit(1); +} + +console.log( + drifted.length + ? `brand-numbers: updated ${drifted.length} file(s)` + : `brand-numbers: already up to date (${everUsed.size} keys in use)`, +); diff --git a/src/router/brand-numbers.test.ts b/src/router/brand-numbers.test.ts new file mode 100644 index 00000000..97d8a6a6 --- /dev/null +++ b/src/router/brand-numbers.test.ts @@ -0,0 +1,105 @@ +/** + * Pins the numbers about ClawRouter that BlockRun publishes but cannot count. + * + * blockrun.ai/brand/numbers.json is the artifact every public repo's marketing + * copy is generated from. Most of it is derived from the model catalog, so it + * cannot go stale. The `clawrouter.*` block is different: those facts live in + * THIS repo's code, so over there they are hand-asserted. + * + * This test is what makes that safe. Change the scorer or the alias map and the + * build fails HERE — in the repo that can actually fix it — instead of quietly + * making a claim in 37 READMEs wrong. + * + * When one fails: confirm the change is intended, update brand-numbers.json, + * and update the same numbers in blockrun's src/app/brand/numbers.json/route.ts. + */ +import { readFileSync } from "node:fs"; + +import { describe, expect, it } from "vitest"; + +import { MODEL_ALIASES } from "../models.js"; +import { DEFAULT_ROUTING_CONFIG } from "./config.js"; +import { classifyByRules } from "./rules.js"; + +const published = JSON.parse(readFileSync("brand-numbers.json", "utf8")) as { + clawrouter: { dimensions: number; tiers: number; profiles: number; aliases: number }; +}; + +describe("numbers BlockRun publishes about ClawRouter", () => { + it("scores across the advertised number of dimensions", () => { + // A prompt with enough substance to take the full scoring path rather than + // an early return — the count is what matters, not the verdict. + const result = classifyByRules( + "Refactor this module to remove the duplicated retry logic and add tests.", + undefined, + 2_000, + DEFAULT_ROUTING_CONFIG.scoring, + ); + expect(result.dimensions).toBeDefined(); + expect(result.dimensions).toHaveLength(published.clawrouter.dimensions); + }); + + it("exposes the advertised number of routing profiles", () => { + // eco / auto / premium / agentic — one tier table each. + const profiles = Object.keys(DEFAULT_ROUTING_CONFIG).filter((k) => + /^(tiers|[a-z]+Tiers)$/.test(k), + ); + expect(profiles).toHaveLength(published.clawrouter.profiles); + }); + + it("exposes the advertised number of tiers, identically in every profile", () => { + const profiles = Object.entries(DEFAULT_ROUTING_CONFIG).filter(([k]) => + /^(tiers|[a-z]+Tiers)$/.test(k), + ); + for (const [name, table] of profiles) { + // A profile missing a tier silently falls back mid-route, so check each + // rather than only the default one. + expect(Object.keys(table as object), name).toHaveLength(published.clawrouter.tiers); + } + }); + + it("ships the advertised number of model aliases", () => { + expect(Object.keys(MODEL_ALIASES)).toHaveLength(published.clawrouter.aliases); + }); +}); + +describe("package.json description", () => { + // npm renders this on the package page, where no marker can reach it — it is + // not markdown, so scripts/sync-brand-numbers.mjs cannot rewrite it. Assert + // instead, so the one un-syncable surface still cannot drift. + const numbers = JSON.parse(readFileSync("brand-numbers.json", "utf8")) as { + models: { chatVisible: number; free: number }; + savings: { autoVsBaselinePct: number }; + }; + const { description } = JSON.parse(readFileSync("package.json", "utf8")) as { + description: string; + }; + + it("quotes the published savings figure", () => { + expect(description).toContain(`save ${numbers.savings.autoVsBaselinePct}% on inference costs`); + }); + + it("quotes the published model counts", () => { + expect(description).toContain( + `${numbers.models.chatVisible} models (${numbers.models.free} free)`, + ); + }); +}); + +describe("README hero badge", () => { + // The free-model count is baked into a shields URL and its alt text. A marker + // cannot go inside an HTML attribute without breaking the tag, so this is the + // other surface sync-brand-numbers.mjs cannot reach. Assert it instead. + const readme = readFileSync("README.md", "utf8"); + const free = ( + JSON.parse(readFileSync("brand-numbers.json", "utf8")) as { models: { free: number } } + ).models.free; + + it("shows the published free-model count", () => { + expect(readme).toContain(`badge/🆓_${free}_Free_Models-success`); + }); + + it("keeps its alt text in step with the badge", () => { + expect(readme).toContain(`alt="${free} free models"`); + }); +}); diff --git a/src/router/config.ts b/src/router/config.ts index 0c67ac30..532b2f3b 100644 --- a/src/router/config.ts +++ b/src/router/config.ts @@ -4,7 +4,7 @@ * All routing parameters as a TypeScript constant. * Operators override via openclaw.yaml plugin config. * - * Scoring uses 14 weighted dimensions with sigmoid confidence calibration. + * Scoring uses 15 weighted dimensions with sigmoid confidence calibration. */ import type { RoutingConfig } from "./types.js"; diff --git a/src/router/rules.ts b/src/router/rules.ts index b3a6ec7d..6e997aad 100644 --- a/src/router/rules.ts +++ b/src/router/rules.ts @@ -1,7 +1,7 @@ /** * Rule-Based Classifier (v2 — Weighted Scoring) * - * Scores a request across 14 weighted dimensions and maps the aggregate + * Scores a request across 15 weighted dimensions and maps the aggregate * score to a tier using configurable boundaries. Confidence is calibrated * via sigmoid — low confidence triggers the fallback classifier. * @@ -145,7 +145,8 @@ export function classifyByRules( // and make every request score identically. See GitHub issue #50. const userText = prompt.toLowerCase(); - // Score all 14 dimensions against user text only + // Score the base dimensions against user text only; the agentic dimension is + // appended below, so the scored total is one more than this list. const dimensions: DimensionScore[] = [ // Token count uses total estimated tokens (system + user) — context size matters for model selection scoreTokenCount(estimatedTokens, config.tokenCountThresholds),