Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,9 @@ jobs:

- name: Build
run: pnpm run build

# Fails when a number in the docs disagrees with brand-numbers.json.
# Offline by design: it reads the committed snapshot and never fetches, so
# a blockrun.ai deploy in progress cannot fail this repo's CI.
- name: Brand numbers
run: node scripts/sync-brand-numbers.mjs --check
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# @blockrun/llm (TypeScript SDK)

> **@blockrun/llm** is a TypeScript/Node.js SDK for accessing 41+ large language models (GPT-5, Claude, Gemini, Grok, DeepSeek, Kimi, and more) with automatic pay-per-request USDC micropayments via the x402 protocol. No API keys required — your wallet signature is your authentication. Supports **streaming**, smart routing, Base and Solana chains.
> **@blockrun/llm** is a TypeScript/Node.js SDK for accessing <!-- br:models.chatVisible -->66<!-- /br:models.chatVisible --> large language models (GPT-5, Claude, Gemini, Grok, DeepSeek, Kimi, and more) with automatic pay-per-request USDC micropayments via the x402 protocol. No API keys required — your wallet signature is your authentication. Supports **streaming**, smart routing, Base and Solana chains.
>
> 🆓 **Includes 7 fully-free NVIDIA-hosted models** (5 visible in `/v1/models`, 2 hidden but directly callable) — DeepSeek V4 Flash (1M context), Nemotron Nano Omni (vision), Qwen3 Coder, Llama 4, Mistral, plus the gpt-oss pair. Zero USDC, no rate-limit gimmicks. Use `routingProfile: 'free'` or call any `nvidia/*` model directly.

Expand Down Expand Up @@ -232,7 +232,7 @@ const client = new LLMClient();
const result = await client.smartChat('What is 2+2?');
console.log(result.response); // '4'
console.log(result.model); // 'moonshot/kimi-k2.5' (cheap, fast)
console.log(`Saved ${(result.routing.savings * 100).toFixed(0)}%`); // 'Saved 78%'
console.log(`Saved ${(result.routing.savings * 100).toFixed(0)}%`); // 'Saved 87%'

// Complex reasoning task -> routes to reasoning model
const complex = await client.smartChat('Prove the Riemann hypothesis step by step');
Expand Down Expand Up @@ -263,7 +263,7 @@ const reply = await client.chat('nvidia/deepseek-v4-flash', 'hello', {

| Profile | Description | Best For |
|---------|-------------|----------|
| `free` | NVIDIA free tier — smart-routes across 8 models (DeepSeek V4 Flash, Nemotron Nano Omni, Qwen3, Llama 4, Mistral, plus 2 hidden gpt-oss) | Zero-cost testing, dev, prod |
| `free` | NVIDIA free tier — smart-routes across <!-- br:models.free -->8<!-- /br:models.free --> models (DeepSeek V4 Flash, Nemotron Nano Omni, Qwen3, Llama 4, Mistral, plus 2 hidden gpt-oss) | Zero-cost testing, dev, prod |
| `eco` | Cheapest models per tier (DeepSeek, xAI) | Cost-sensitive production |
| `auto` | Best balance of cost/quality (default) | General use |
| `premium` | Top-tier models (OpenAI, Anthropic) | Quality-critical tasks |
Expand Down Expand Up @@ -858,7 +858,7 @@ const response2 = await client.chat('anthropic/claude-sonnet-4', 'Write a haiku'

### Smart Routing (ClawRouter)

Save up to 78% on inference costs with intelligent model routing. ClawRouter uses a 14-dimension rule-based scoring algorithm to select the cheapest model that can handle your request (<1ms, 100% local).
Save up to <!-- br:savings.autoVsBaselinePct -->87<!-- /br:savings.autoVsBaselinePct -->% on inference costs with intelligent model routing. ClawRouter uses a <!-- br:clawrouter.dimensions -->15<!-- /br:clawrouter.dimensions -->-dimension rule-based scoring algorithm to select the cheapest model that can handle your request (<1ms, 100% local).

```typescript
import { LLMClient } from '@blockrun/llm';
Expand All @@ -870,7 +870,7 @@ const result = await client.smartChat('What is 2+2?');
console.log(result.response); // '4'
console.log(result.model); // 'google/gemini-2.5-flash'
console.log(result.routing.tier); // 'SIMPLE'
console.log(`Saved ${(result.routing.savings * 100).toFixed(0)}%`); // 'Saved 78%'
console.log(`Saved ${(result.routing.savings * 100).toFixed(0)}%`); // 'Saved 87%'

// Routing profiles
const free = await client.smartChat('Hello!', { routingProfile: 'free' }); // Zero cost
Expand All @@ -883,7 +883,7 @@ const premium = await client.smartChat('Write a legal brief', { routingProfile:

| Profile | Description | Best For |
|---------|-------------|----------|
| `free` | NVIDIA free tier (9 models, smart-routed) | Zero-cost testing, dev, prod |
| `free` | NVIDIA free tier (<!-- br:models.free -->8<!-- /br:models.free --> models, smart-routed) | Zero-cost testing, dev, prod |
| `eco` | Budget-optimized | Cost-sensitive workloads |
| `auto` | Intelligent routing (default) | General use |
| `premium` | Best quality models | Critical tasks |
Expand Down Expand Up @@ -1427,7 +1427,7 @@ The `AnthropicClient` wraps the official `@anthropic-ai/sdk` with a custom fetch
When you make an API call, the SDK automatically handles x402 payment. It signs a USDC transaction locally using your wallet private key (which never leaves your machine), and includes the payment proof in the request header. Settlement is non-custodial and instant on Base or Solana.

### What is smart routing / ClawRouter?
ClawRouter is a built-in smart routing engine that analyzes your request across 14 dimensions and automatically picks the cheapest model capable of handling it. Routing happens locally in under 1ms. It can save up to 78% on LLM costs compared to using premium models for every request.
ClawRouter is a built-in smart routing engine that analyzes your request across <!-- br:clawrouter.dimensions -->15<!-- /br:clawrouter.dimensions --> dimensions and automatically picks the cheapest model capable of handling it. Routing happens locally in under 1ms. It can save up to <!-- br:savings.autoVsBaselinePct -->87<!-- /br:savings.autoVsBaselinePct -->% on LLM costs compared to using premium models for every request.

### Does it support streaming?
Yes — as of v1.6.1. Use `client.chatCompletionStream()` for native streaming or `stream: true` in the OpenAI-compatible client. Payment is handled automatically: the SDK signs USDC payment before streaming begins, and caches payment requirements per model so subsequent calls skip the 402 round-trip (~200ms faster).
Expand Down
34 changes: 34 additions & 0 deletions brand-numbers.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"$schema": "https://blockrun.ai/brand/numbers.schema.json",
"version": 1,
"models": {
"chatVisible": 66,
"totalVisible": 86,
"free": 8,
"freeWithheld": 17,
"image": 8,
"video": 5,
"music": 1,
"speech": 5,
"soundfx": 1,
"withFallback": 44,
"withFallbackAllEntries": 75
},
"clawrouter": {
"dimensions": 15,
"tiers": 4,
"profiles": 4,
"aliases": 202
},
"mcp": {
"tools": 19
},
"chains": {
"rpc": 40
},
"savings": {
"baselineModel": "anthropic/claude-opus-5",
"ecoVsBaselinePct": 98,
"autoVsBaselinePct": 87
}
}
256 changes: 256 additions & 0 deletions scripts/sync-brand-numbers.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,256 @@
#!/usr/bin/env node
/**
* Sync marketing numbers from BlockRun's canonical brand artifact.
*
* This file is copied byte-for-byte into every public repo as
* scripts/sync-brand-numbers.mjs. It is a copy rather than an npm package on
* purpose: a package would mean 37 dependency bumps, and several consuming
* repos have no package.json at all. Zero dependencies, plain Node.
*
* node scripts/sync-brand-numbers.mjs rewrite markers in place
* node scripts/sync-brand-numbers.mjs --check exit 1 on drift, write nothing
* node scripts/sync-brand-numbers.mjs --refresh re-fetch the artifact first
*
* --check NEVER touches the network. PR CI must be deterministic and offline:
* if it fetched, a deploy in progress would fail every repo in the org at once.
* Freshness is the fan-out job's problem, not the pull request's.
*
* Markers look like: <!-- br:models.chatVisible -->66<!-- /br:models.chatVisible -->
* and wrap the WHOLE token, so a badge URL, its alt text and the prose number
* can all regenerate from one key.
*/
import { readFileSync, writeFileSync, readdirSync, statSync } from "node:fs";
import { join, relative, extname } from "node:path";

const ROOT = process.cwd();
const SNAPSHOT = join(ROOT, "brand-numbers.json");
// ORIGIN is tried first because it IS the truth — the mirror can only ever be
// as fresh as the last time someone refreshed it. The mirror exists so a repo
// can still sync while blockrun.ai is down, not to front the origin.
//
// The mirror is awesome-blockrun's own brand-numbers.json: that repo consumes
// the artifact like every other, and its snapshot doubles as the org's copy.
// One file, one role per repo, nothing to keep in step by hand.
const ORIGIN = "https://blockrun.ai/brand/numbers.json";
const MIRROR =
"https://raw.githubusercontent.com/BlockRunAI/awesome-blockrun/main/brand-numbers.json";

const argv = new Set(process.argv.slice(2));
const check = argv.has("--check");
const refresh = argv.has("--refresh");

const SKIP_DIRS = new Set([
"node_modules", ".git", "dist", "build", "out", ".next", "coverage",
"vendor", "target", "__pycache__", ".venv", "venv",
]);
const TEXT_EXT = new Set([".md", ".mdx"]);

/* ── 1. numbers ──────────────────────────────────────────────────────────── */

async function loadNumbers() {
if (!refresh) {
try {
return JSON.parse(readFileSync(SNAPSHOT, "utf8"));
} catch {
fail(
`no brand-numbers.json in ${ROOT}\n` +
` run with --refresh once to seed it from ${ORIGIN}`,
);
}
}
for (const url of [ORIGIN, MIRROR]) {
try {
const res = await fetch(url, { signal: AbortSignal.timeout(10_000) });
if (!res.ok) continue;
const json = await res.json();
writeFileSync(SNAPSHOT, `${JSON.stringify(json, null, 2)}\n`);
return json;
} catch {
/* try the next source */
}
}
fail(`could not refresh from ${MIRROR} or ${ORIGIN}`);
}

/** Flatten nested numbers into dotted keys, ignoring $comment / rationale prose. */
function flatten(obj, prefix = "") {
return Object.entries(obj).flatMap(([k, v]) => {
if (k.startsWith("$")) return [];
const key = `${prefix}${k}`;
if (v && typeof v === "object" && !Array.isArray(v)) return flatten(v, `${key}.`);
if (v === null) return [];
return [[key, v]];
});
}

/* ── 2. renderers ────────────────────────────────────────────────────────── */

/**
* How a key becomes text. Default is the bare value.
*
* A marker may carry an `@modifier` — `<!-- br:mcp.tools@badge -->` — which
* selects a renderer without changing which number is looked up. The modifier
* is what makes a key reusable: the same mcp.tools appears as a shields badge
* at the top of a README and as a bare "19 tools" in a table two screens down,
* and one marker still keeps the badge URL, its alt text and the label in step.
*
* Renderers are registered under the FULL marker name so a badge's label is
* written out rather than guessed from the key.
*/
const badge = (label) => (n) =>
`<img src="https://img.shields.io/badge/${label}-${n}-5B9BF6?style=flat-square&labelColor=0B0A0F" alt="${n} ${label}">`;

const RENDER = {
"mcp.tools@badge": badge("tools"),
"models.totalVisible@badge": badge("models"),
"models.chatVisible@badge": badge("models"),
};
const render = (marker, value) => (RENDER[marker] ?? String)(value);

/** `mcp.tools@badge` looks up `mcp.tools`. Unmodified markers are unaffected. */
const keyOf = (marker) => marker.split("@")[0];

/* ── 3. marker rewriting ─────────────────────────────────────────────────── */

const esc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const OPEN_ANY = /<!--\s*br:([A-Za-z0-9_.@]+)\s*-->/g;
const CLOSE_ANY = /<!--\s*\/br:([A-Za-z0-9_.@]+)\s*-->/g;

/** Byte ranges of fenced code blocks — markers inside them are documentation. */
function fencedRanges(text) {
const ranges = [];
const fence = /^(\s*)(`{3,}|~{3,})[^\n]*$/gm;
let open = null;
for (let m; (m = fence.exec(text)); ) {
if (open === null) open = m.index;
else {
ranges.push([open, m.index + m[0].length]);
open = null;
}
}
return ranges;
}

function syncFile(file, numbers, problems) {
const before = readFileSync(file, "utf8");
const rel = relative(ROOT, file);
const fenced = fencedRanges(before);
const inFence = (i) => fenced.some(([a, b]) => i >= a && i < b);
const known = new Map(numbers);
const used = new Set();

// Markers actually present, so a file is only ever rewritten for what it uses
// and an @modifier is carried through to the renderer verbatim.
const markers = new Set();
// A marker naming a key that does not exist is an error, never a silent
// no-op: a typo'd marker would otherwise sit there looking synced forever.
for (const [re, shown] of [
[OPEN_ANY, (n) => `<!-- br:${n} -->`],
[CLOSE_ANY, (n) => `<!-- /br:${n} -->`],
]) {
for (const m of before.matchAll(re)) {
if (inFence(m.index)) continue;
markers.add(m[1]);
if (!known.has(keyOf(m[1]))) problems.push(`${rel}: unknown key ${shown(m[1])}`);
}
}

let after = before;
for (const marker of markers) {
const key = keyOf(marker);
if (!known.has(key)) continue;
const value = known.get(key);
const pair = new RegExp(
`(<!--\\s*br:${esc(marker)}\\s*-->)([\\s\\S]*?)(<!--\\s*/br:${esc(marker)}\\s*-->)`,
"g",
);
after = after.replace(pair, (whole, open, inner, close, offset) => {
if (inFence(offset)) return whole;
// Nesting means the closing tag of an inner marker would be consumed by
// the outer one. Refuse rather than produce mangled output.
if (/<!--\s*\/?br:/.test(inner)) {
problems.push(`${rel}: nested marker inside br:${marker}`);
return whole;
}
used.add(marker);
return open + render(marker, value) + close;
});

// An opening tag with no partner silently swallows the rest of the file on
// a naive regex, so catch it explicitly.
const opens = [...before.matchAll(new RegExp(`<!--\\s*br:${esc(marker)}\\s*-->`, "g"))]
.filter((m) => !inFence(m.index)).length;
const closes = [...before.matchAll(new RegExp(`<!--\\s*/br:${esc(marker)}\\s*-->`, "g"))]
.filter((m) => !inFence(m.index)).length;
if (opens !== closes) problems.push(`${rel}: unbalanced marker br:${marker} (${opens} open, ${closes} close)`);
}

return { before, after, changed: before !== after, used };
}

/* ── 4. walk ─────────────────────────────────────────────────────────────── */

function* walk(dir) {
for (const name of readdirSync(dir)) {
if (SKIP_DIRS.has(name)) continue;
const p = join(dir, name);
const s = statSync(p);
if (s.isDirectory()) yield* walk(p);
else if (TEXT_EXT.has(extname(name))) yield p;
}
}

function fail(msg) {
console.error(`brand-numbers: ${msg}`);
process.exit(1);
}

/* ── 5. run ──────────────────────────────────────────────────────────────── */

const raw = await loadNumbers();
const numbers = flatten(raw);
const problems = [];
const drifted = [];
const everUsed = new Set();

for (const file of walk(ROOT)) {
const { before, after, changed, used } = syncFile(file, numbers, problems);
used.forEach((k) => everUsed.add(k));
if (!changed) continue;
drifted.push({ file: relative(ROOT, file), before, after });
if (!check) writeFileSync(file, after);
}

if (problems.length) {
for (const p of problems) console.error(` ${p}`);
fail(`${problems.length} marker problem(s)`);
}

if (check) {
if (drifted.length === 0) {
console.log(`brand-numbers: up to date (${everUsed.size} keys in use)`);
process.exit(0);
}
console.error("brand-numbers: these files disagree with brand-numbers.json\n");
for (const { file, before, after } of drifted) {
const b = before.split("\n");
const a = after.split("\n");
for (let i = 0; i < Math.max(b.length, a.length); i++) {
if (b[i] !== a[i]) {
console.error(` ${file}:${i + 1}`);
console.error(` - ${(b[i] ?? "").trim()}`);
console.error(` + ${(a[i] ?? "").trim()}`);
}
}
}
console.error(
"\n fix with: node scripts/sync-brand-numbers.mjs && git commit -am 'chore: sync brand numbers'",
);
process.exit(1);
}

console.log(
drifted.length
? `brand-numbers: updated ${drifted.length} file(s)`
: `brand-numbers: already up to date (${everUsed.size} keys in use)`,
);
Loading