Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -482,8 +482,25 @@ Background relays these via `chrome.runtime.sendMessage` to the side panel, whic

The action-mode intent gate runs before the first browser tool call. Off uses the compact schema; Try and Strict use the full planning schema, with unset storage defaulting to Try. The full planner prompt requires a single JSON object with summary, concrete steps, validated `skill_ids`, memory strategy, scheduling hint, risks, and an action mode. Mid/Full planners receive only the eligible routing catalog, and approved skill IDs are activated before the normal execution model call. `normalizePlan()` bounds and sanitizes each field; `formatPlanMarkdown()` renders the side-panel review card; `formatPlanScratchpad()` pins the approved or edited plan as an `[Approved plan]` scratchpad entry.

The browser-owned per-turn runtime context includes the effective
`runtime_mode` and whether mutation tools are enabled. This envelope is added
once to the current user turn and is shared by the planner and executor, so
page content or stale conversation history cannot redefine the live mode. For
Act/Dev runs it also directs missing required values to `clarify` after useful
inspection; planner guidance treats `done` as terminal, never as a way to ask
for information needed to continue.

Planner calls are traced with `phase: "planner"` when trace recording is enabled. They also use the cost allowance guard, abort checks, a JSON-repair retry, and Qwen/DeepSeek no-think handling. A failed repair cannot authorize actions: Try falls back to an Ask/read-only turn, while Strict stops.

LLM-request trace events include privacy-safe prompt provenance: the controlled
prompt variant, system-prompt and aggregate message character counts, message
role counts, declared prompt/tool policy revisions, and structured checks
comparing the system prompt and runtime envelope with the effective run mode.
Raw system-prompt text, message text, tool schemas, and tool names are neither
copied nor fingerprinted in request events. Policy revisions are bumped when
controlled prompt templates or tool-exposure rules change; private request
content does not affect them.

Each new trace run records the manifest version that created it. `/export`
Markdown records the exporting version, `/export --traces` records both the
exporting version and every turn's recording version, and Traces-page JSON adds
Expand Down
11 changes: 9 additions & 2 deletions docs/export-and-workflow-formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ workflow. These files have different privacy and compatibility properties.
| Command or UI | File | Format | Treat as sensitive? |
|---|---|---|---|
| `/export` | `webbrain-chat-<timestamp>.md` | Conversation Markdown | Yes. It contains visible chat and system messages. |
| `/export --traces` | `webbrain-traces-<timestamp>.md` | Recorded tool-chain Markdown | Yes. It can contain prompts, model output, tool arguments, URLs, and results. |
| `/export --traces` | `webbrain-traces-<timestamp>.md` | Recorded tool-chain Markdown | Yes. It can contain user prompts, model output, tool arguments, URLs, and results. Raw system prompts are not embedded. |
| `/export --config` | `webbrain-config-<timestamp>.json` | `webbrain-config/1` | **Yes. It is plaintext and can contain API keys, profile data, and user memory.** |
| `/workflow --export <id>` | `<name>.webbrain-workflow.json` | `webbrain-workflow/1` | Review before sharing. Runtime values are omitted, but saved targets and URL scopes remain. |
| Traces page **Export JSON** | `webbrain-trace-<model>-<run-id>.json` | `webbrain-trace/1` | Yes. It contains the raw recorded run and may include screenshots. |
Expand Down Expand Up @@ -36,7 +36,14 @@ the exporting extension version but has no schema identifier.
`/export --traces` exports recorded runs associated with the current
conversation. Tracing must have been enabled when the runs occurred. Each turn
contains its recording version when available, model and status metadata, model
responses, tool calls, arguments, and rendered results.
responses, tool calls, arguments, rendered results, the allowlisted runtime
snapshot, and privacy-safe prompt provenance. Provenance identifies the
controlled prompt variant and records character counts plus declared prompt and
tool policy revisions. It also reports whether the prompt/runtime envelope
matched the effective mode; it does not embed or fingerprint raw system-prompt
text, message text, tool schemas, or tool names. Policy revisions identify the
controlled code path and are bumped when its prompt or tool-exposure rules
change; they do not vary with private request content.

Screenshots, vision sub-calls, and internal trace notes are omitted from this
Markdown format. The export may be marked partial or truncated when the browser
Expand Down
8 changes: 7 additions & 1 deletion docs/privacy-and-data-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,11 +120,17 @@ the stored copies are not separately synced to WebBrain.
When enabled (Settings → Display → "Record traces"), every agent run is written to an IndexedDB database (`webbrain_traces`):

- **`runs` store**: model, provider, token totals, timestamps, user message, final content
- **`events` store**: per-step LLM requests/responses, tool calls with args and results
- **`events` store**: per-step LLM request provenance, model responses, and tool calls with args and results. Request provenance contains counts, controlled prompt/mode labels, and declared prompt/tool policy revisions; it neither duplicates nor fingerprints raw system prompts, message text, tool schemas, or tool names.
- **`shots` store**: screenshot blobs

The Traces page (`ui/traces.html`) reads from local IndexedDB only. Export produces a JSON blob saved to the user's Downloads folder. **No trace data ever leaves the browser.**

Each run also records an allowlisted effective runtime snapshot (including mode
and prompt tier). Trace Markdown surfaces that snapshot and the privacy-safe
request provenance so mode/prompt mismatches can be diagnosed without exporting
the full private prompt payload. Policy revisions change with controlled
prompt/tool rules, not with private request content.

### Saved Workflows

`/workflow --save <name>` locally compiles the latest successful trace into a
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "webbrain",
"version": "28.1.2",
"version": "28.1.3",
"description": "Open-source AI browser agent — chat with pages, automate tasks, multi-provider LLM support.",
"private": true,
"type": "module",
Expand Down
9 changes: 7 additions & 2 deletions src/chrome/ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# WebBrain Chrome/Edge Extension — Architecture

> Version 28.1.2 · Manifest V3 · Service Worker background
> Version 28.1.3 · Manifest V3 · Service Worker background

## High-Level Overview

Expand Down Expand Up @@ -691,10 +691,15 @@ in-progress Markdown after older delta events have been acknowledged.
Off by default. Enabled via Settings → Display → "Record traces". When on, every agent run writes to an IndexedDB database (`webbrain-traces`):

- `runs` store: one row per user message — model, provider, token totals, timestamps.
- `events` store: one row per LLM request/response, tool call, screenshot. Rows are indexed by `(runId, seq)`.
- `events` store: one row per LLM request/response, tool call, screenshot. LLM requests retain content-free prompt provenance (controlled variant, counts, declared prompt/tool policy revisions, and runtime-mode alignment), not fingerprints or raw system prompts, message text, tool schemas, or tool names. Policy revisions are bumped when controlled prompt templates or tool-exposure rules change; private request content does not affect them. Rows are indexed by `(runId, seq)`.

The Traces page (`ui/traces.html`) lists runs and renders their event timelines. Exporting produces a JSON blob identical to the ones used in this session's debugging. Data never leaves the machine — this is why `unlimitedStorage` is requested (a multi-step run with screenshots is 1–10 MB).

The browser-owned trusted runtime context also carries the effective mode once
per run. Both planner and executor receive the same envelope. Act/Dev envelopes
advertise mutation availability and route still-missing required inputs through
`clarify`; `done` remains terminal.

---

## Display Settings
Expand Down
2 changes: 1 addition & 1 deletion src/chrome/manifest.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"manifest_version": 3,
"name": "WebBrain",
"version": "28.1.2",
"version": "28.1.3",
"description": "Open-source AI browser agent — chat with pages, automate tasks, multi-provider LLM support.",
"permissions": [
"sidePanel",
Expand Down
81 changes: 75 additions & 6 deletions src/chrome/src/agent/agent.js
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ import {
workflowUrlMatches,
} from './workflows.js';
import { mergeRedactionFrameRegions, mapRegionsToImage, pixelateDataUrl } from './screenshot-redaction.js';
import { buildTrustedRuntimeContext, stripTrustedRuntimeContext } from './runtime-context.js';
import { buildTrustedRuntimeContext, replaceTrustedRuntimeMode, stripTrustedRuntimeContext } from './runtime-context.js';
import {
isSelectionProseAction,
normalizeSelectionAction,
Expand Down Expand Up @@ -3680,7 +3680,9 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
// Dynamic trusted state belongs in the per-turn user context, not the
// cache-stable system prompt. The same enriched message is passed to the
// planner gate and the main agent loop, so neither has to guess the clock.
let contextLine = `${buildTrustedRuntimeContext()}\n\n`;
let contextLine = `${buildTrustedRuntimeContext({
runtimeMode: this._effectiveRunMode(tabId),
})}\n\n`;

// Collect URL + title via chrome.tabs (cheap, no debugger needed).
let url = '';
Expand Down Expand Up @@ -9430,6 +9432,19 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
if (messages[0]?.role === 'system') {
messages[0].content = this._buildSystemPrompt('ask', tabId);
}
const currentUser = [...messages].reverse().find(message => (
message?.role === 'user'
&& this._messageText(message.content).includes('[Trusted runtime context')
));
if (typeof currentUser?.content === 'string') {
currentUser.content = replaceTrustedRuntimeMode(currentUser.content, 'ask');
} else if (Array.isArray(currentUser?.content)) {
currentUser.content = currentUser.content.map(block => (
typeof block?.text === 'string'
? { ...block, text: replaceTrustedRuntimeMode(block.text, 'ask') }
: block
));
}
this._persist(tabId);
return 'ask';
}
Expand Down Expand Up @@ -9615,6 +9630,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
toolsCount: 0,
...Agent._traceMediaCounts(messages),
phase: 'read_scope',
}, {
messages,
tools: [],
runtimeMode: this._effectiveRunMode(tabId),
});
} catch {}
}
Expand Down Expand Up @@ -9644,6 +9663,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
let readScope = parseReadScopeFromContent(result.content);
if (!readScope) {
onUpdate('thinking', { step: 0, note: 'Checking conversation scope… retrying JSON output' });
const repairMessages = this._readScopeRepairMessages(messages);
if (runId) {
try {
trace.recordLLMRequest(runId, 0, {
Expand All @@ -9653,13 +9673,17 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
toolsCount: 0,
phase: 'read_scope',
repair: true,
}, {
messages: repairMessages,
tools: [],
runtimeMode: this._effectiveRunMode(tabId),
});
} catch {}
}
const repairStartedAt = Date.now();
result = await this._chatWithCostAllowance(
provider,
this._readScopeRepairMessages(messages),
repairMessages,
{ ...this._plannerChatOptions(provider, true, true), temperature: 0 },
costState,
{ tabId, generationName: 'read_scope' },
Expand Down Expand Up @@ -9742,6 +9766,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
toolsCount: 0,
...Agent._traceMediaCounts(plannerMessages),
phase: 'intent',
}, {
messages: plannerMessages,
tools: [],
runtimeMode: this._effectiveRunMode(tabId),
});
} catch {}
}
Expand Down Expand Up @@ -9891,6 +9919,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
toolsCount: 0,
...Agent._traceMediaCounts(plannerMessages),
phase: 'planner',
}, {
messages: plannerMessages,
tools: [],
runtimeMode: this._effectiveRunMode(tabId),
});
} catch {}
}
Expand Down Expand Up @@ -10346,6 +10378,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
toolsCount: Array.isArray(tools) ? tools.length : 0,
...Agent._traceMediaCounts(prunedMessages),
phase,
}, {
messages: prunedMessages,
tools: Array.isArray(tools) ? tools : [],
runtimeMode: this._effectiveRunMode(tabId),
});
} catch {}
}
Expand Down Expand Up @@ -13895,6 +13931,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
? (carried.successfulRequiredSchedulingToolCalls || 0)
: 0,
recoveryAttempted: false,
runtimeModeCorrectionAttempted: false,
staleCancellationRecoveryAttempted: false,
};
this._planExecutionGuards.set(tabId, state);
Expand Down Expand Up @@ -14091,8 +14128,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
&& String(object.mode || '').toLowerCase() !== 'inactive';
if (plannerShape || policyShape) return state.allowsPlannerShapedResult !== true;
}
const runtimeModeContradiction = /\b(?:switch|change|set)\s+(?:back\s+)?to\s+act\s+mode\b|\b(?:currently|still|now)\s+(?:running\s+)?in\s+ask\s+mode\b/i.test(text);
if (runtimeModeContradiction) return true;
if (this._isRuntimeModeContradictionTerminal(text)) return true;
// "Next, I will …" / "I plan to …" is agent-continue language and is always
// invalid as a terminal. Bare "I will …" is evidence-gated so drafted reply
// text can finish after a real task tool without a planner exemption flag.
Expand All @@ -14108,11 +14144,16 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
return false;
}

_isRuntimeModeContradictionTerminal(content) {
return /\b(?:switch|change|set)\s+(?:back\s+)?to\s+act\s+mode\b|\b(?:currently|still|now)\s+(?:running\s+)?in\s+ask\s+mode\b/i.test(String(content || ''));
}

_planOnlyTerminalDecision(tabId, content, { viaDone = false, outcome = null } = {}) {
const state = this._planExecutionGuards.get(tabId);
if (!state?.enabled) return null;
if (!viaDone && this._isSafetyRefusalTerminal(content)) return null;
const terminalFailure = viaDone && (outcome === 'partial' || outcome === 'failed');
const runtimeModeContradiction = this._isRuntimeModeContradictionTerminal(content);
// A structured failure may naturally say "I will need credentials".
// Ignore only that prose-promise heuristic; explicit planner/policy shapes
// and plan headings remain invalid even for failed/partial done calls.
Expand All @@ -14132,6 +14173,17 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
const invalidPlainFinal = !viaDone;
const invalidDone = viaDone && (looksPlanOnly || missingEvidence);
if (!invalidPlainFinal && !invalidDone) return null;
if (runtimeModeContradiction
&& !missingRequiredSchedulingTool
&& !state.runtimeModeCorrectionAttempted) {
state.recoveryAttempted = true;
state.runtimeModeCorrectionAttempted = true;
return {
retry: true,
retryAssistantContent: null,
nudge: '[RUNTIME MODE CORRECTION: The trusted runtime for this run is Act/Dev, not Ask mode. Page-changing tools are available. Continue the authorized task with the permitted tools now. If a required value is missing, call clarify without modifying that field. If a genuine blocker remains, call done with outcome partial or failed and explain that blocker without claiming the run is in Ask mode.]',
};
}
if (!state.recoveryAttempted) {
state.recoveryAttempted = true;
state.staleCancellationRecoveryAttempted = staleCancellation;
Expand All @@ -14154,6 +14206,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
};
}
const hasSuccessfulToolEvidence = state.successfulTaskToolCalls > 0;
const hasSuccessfulConsequentialEvidence = state.successfulConsequentialToolCalls > 0;
if (staleCancellation && state.staleCancellationRecoveryAttempted) {
return {
failure: hasSuccessfulToolEvidence
Expand All @@ -14162,9 +14215,21 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
status: 'plan_only_output',
};
}
if (runtimeModeContradiction) {
return {
failure: hasSuccessfulConsequentialEvidence
? 'The model still claimed Ask mode after a runtime-mode correction even though this was an Act/Dev run. Some consequential tools may already have completed, but final completion was not verified. Inspect the current page before retrying to avoid duplicate side effects.'
: hasSuccessfulToolEvidence
? 'The model still claimed Ask mode after a runtime-mode correction instead of using the available Act/Dev tools. Only read-only task evidence was recorded; no consequential page action was recorded, and nothing was verified as changed, submitted, or sent.'
: 'The model still claimed Ask mode after a runtime-mode correction instead of using the available Act/Dev tools. No successful page action was verified, and nothing was verified as changed, submitted, or sent.',
status: 'plan_only_output',
};
}
return {
failure: hasSuccessfulToolEvidence
failure: hasSuccessfulConsequentialEvidence
? 'Some task tools completed, but I could not verify a valid completion after the recovery attempt. Please inspect the current page before retrying to avoid duplicate side effects.'
: hasSuccessfulToolEvidence
? 'Some read-only task tools completed, but I could not verify the requested action after the recovery attempt. No consequential page action was recorded, and nothing was verified as changed, submitted, or sent.'
: 'I could not verify any requested page action after the recovery attempt, so I stopped without claiming completion. No successful action was verified, and nothing was verified as submitted or sent.',
status: 'plan_only_output',
};
Expand Down Expand Up @@ -23479,6 +23544,10 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
messageCount: prunedMessages.length,
toolsCount: (chatOpts.tools || []).length,
...Agent._traceMediaCounts(prunedMessages),
}, {
messages: prunedMessages,
tools: chatOpts.tools || [],
runtimeMode: mode,
});
if (shouldOrderInteractiveAskTrace) queueAskStreamingTraceWrite(writeRequestTrace);
else writeRequestTrace();
Expand Down
Loading
Loading