diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 996b41b..2b5f720 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -88,5 +88,8 @@ jobs: print("All frontmatter valid.") PY + - name: Support-bundle unit tests + run: node skills/support/scripts/test-build-bundle.js + - name: Brand and guardrail lint run: bash scripts/brand-check.sh diff --git a/.gitignore b/.gitignore index 73328b6..4085744 100644 --- a/.gitignore +++ b/.gitignore @@ -6,6 +6,10 @@ .agent-mail/ .a5c/ +# Fetched service-config cache. Carries internal Taboola infrastructure +# hostnames and datasource credentials — this repo is public. +.claude/transcript-uploader/ + # OS .DS_Store Thumbs.db @@ -24,7 +28,20 @@ venv/ .vscode/ *.swp -# Local PR / review / reply drafts at repo root (not for upstream) +# Local PR / review / reply drafts at repo root (not for upstream). +# Patterns are deliberately loose on the tail: the earlier +# /pr*-comment-*.txt and /reply-to-*.txt required a trailing segment and +# so missed pr2-readme-comment.txt and reply-mcp-*-feedback.txt. /pr-description-*.txt -/pr*-comment-*.txt -/reply-to-*.txt +/pr*-comment*.txt +/reply-*.txt + +# Local working data — eval inputs, hypothesis verdicts, dataset scratch. +# Routinely holds real customer campaign data (account and campaign IDs, +# spend, performance). This repo is public, so one `git add -A` turns a +# messy working tree into a disclosure. Keep it untracked. +/inputs/ + +# Support bundles, wherever they land. build-bundle.js already refuses to +# write inside a git work tree; this is the second line, not the first. +realize-support-*.md diff --git a/CHANGELOG.md b/CHANGELOG.md index d60481c..f16beac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,45 @@ All notable changes to this plugin will be documented here. Format loosely follo - **Plugin renamed** — `name` in both manifests (`.claude-plugin/plugin.json`, `.codex-plugin/plugin.json`) changed from `realize-ads-api` to `realize-plugin`, and a `displayName` of `Realize Plugin` added. Install commands, the skill-invocation namespace (`/realize-plugin:`), and doc titles updated to match. The GitHub repository name (`taboola/realize-claude-plugin`) and all repo URLs are unchanged. `Realize Plugin` is the approved short form of the "Realize MCP and Skills Plugin" naming (drops the Claude-specific framing since the plugin targets Codex too). ### Added +- **Support bundle: a `Summary` section shaped for Salesforce case intake.** Requested by Professional Services after first review of the bundle. Case intake copies an email's subject into the case **Subject** and its body into the case **Description**, so the bundle now renders a copy-ready subject line and a `## 1. Summary` section intended to be pasted as the email body, opening with the requested prolog *"This case has been created by the Realize Plugin, please find the interaction summary below."* + - **The subject is now the user's own complaint**, collapsed to one line, capped at 120 characters, with the first account ID appended for triage. This removes the last model-authored field in the bundle — previously the model wrote a case title *about* the user's problem, which is strictly worse evidence than the user's own sentence. Backticks are stripped so the copy fence can't close early; newlines are collapsed because a break truncates a real subject header. + - **The Summary reports which tools and knowledge the plugin actually consulted** — Realize tools called, skills invoked, and knowledge files read (`knowledge/`, `os/`, `agents/`, `skills/**/SKILL.md`, `skills/**/references/`). This is what PS asked for and it answers a question the bundle previously could not: whether the plugin read the right guidance and still got it wrong, or never read it at all. Those are different bugs. Attribution is extracted from tool calls, never self-reported. + - **The Summary is mechanical, and says so in the file.** PS's request could have been satisfied with a model-written narrative of the session; it deliberately was not. The bundle exists for cases where the plugin misread the request, and a narrative composed by that same plugin reproduces the misreading in the one document meant to expose it. §1 carries counts, tool names, and file paths only, and closes with a line stating that and pointing at the verbatim transcript. + - Sections renumbered: `1. Summary`, `2. At a glance`, `3. What the user reported`, `4. Failed actions`, `5. Realize actions attempted`, `6. Full transcript`. `--title-file` is now the fallback for a run with no user description rather than the normal path. + - **The complaint is now redacted before it becomes the subject.** It stays the user's own wording — `redact()` only removes credential-shaped values, and its flat rule is `=`-only with a length floor so prose is untouched — but users paste error output they never read, and that text now travels into an email subject line. Applies to Section 3 as well. + - **Subject truncation no longer deletes the complaint.** `account_id` is an opaque API string with no length bound; with a long one, `slice(0, MAX - suffix.length - 1)` went negative and sliced from the *end*, replacing the user's entire sentence with a bare `…` in the subject describing their own problem. A `MIN_SUBJECT_TEXT_CHARS` floor now drops the account suffix instead — Section 2 still lists every account acted on. + - **`knowledgeRef` is anchored to the plugin root.** Unanchored, a user's own `~/Documents/os/notes.md` was reported as plugin guidance. A false entry is worse than none here: the section exists to tell "read the right guidance and still got it wrong" from "never read it", and a wrong entry answers that question incorrectly. + - A recorded `Skill` call is no longer also counted under "Other tools used", where it double-reported the same invocation. + - Test suite grows from 36 to **77 assertions** covering subject derivation (including that `$12.40` survives, that a pasted `Bearer` token does not, and that "Secret: Summer Sale" prose is not shredded), the over-long `account_id` regression, `knowledgeRef` root anchoring and its deliberate exclusions, skill/knowledge attribution through `analyze`, and a structural check that the rendered Summary contains no free prose line at all — asserted by line shape rather than a keyword blacklist, so a future edit can't slip a narrative past it with different phrasing. + +- **`support` skill — user-driven support escalation, invoked as `/realize-plugin:support`.** Users run the plugin in their own terminal, so Taboola Professional Services has no visibility into these conversations; a "the plugin gave me the wrong number" report previously arrived with no evidence attached. The command packages the session into a single Markdown file the user emails to `Support@taboola.com`. Contents: a suggested case title, an at-a-glance diagnostic table (session ID, timestamps, turn counts, accounts/campaigns/items involved, actions attempted, **failed count**, plugin + Claude Code version, platform), the user's complaint verbatim, every failed Realize action with its parameters and error text, an ordered table of all actions attempted, and the full rendered transcript. Everything above the transcript is mechanically extracted by `skills/support/scripts/build-bundle.js` — deliberately never model-authored, so a plugin that misread the user can't carry that misreading into its own bug report. + - **Local-only by design.** The script writes one file (Desktop → Downloads → home) and prints the path. There is no upload path; transmitting is the user's decision. + - **Preview-then-confirm**, mirroring the `manage-campaigns` write gate: `--preview` reports what would be captured (including size and the account IDs involved) and writes nothing until the user confirms. + - **Redaction**: bearer tokens, `access_token` / `refresh_token` / `id_token`, `api_key`, `password`, `client_secret`, and JWTs are stripped — in both JSON (`"password": "…"`) and flat (`password=…`) form, since the actions table renders tool inputs as `key=value` where a JSON-shaped pattern never fires. Redaction runs *before* any length-shortening, so a credential can't be cut below the length the patterns match on. `account_id`, `campaign_id`, and `item_id` are deliberately **preserved** — PS cannot reproduce an issue without them. + - **User text is passed by file (`--complaint-file` / `--title-file`), never as a shell argument.** Under a quoted shell argument the complaint is rewritten before it is ever read: `$12.40` arrives as `2.40`, `$500` as `00`, and backticks or `$(…)` execute. Currency is ubiquitous in this domain and users paste error text they didn't author, so the evidence has to bypass the shell entirely. + - **Counts reflect the conversation, not the log.** The transcript stores tool results as `user` records and splits one reply across many `assistant` records, so naive counting overstated a 7-message exchange as 63 user turns and 51 replies. User messages now require a text block; replies key off `stop_reason: "end_turn"`; tool results are reported on their own line. + - **Failure is legible.** `main()` is wrapped so a write error yields a plain-language message and exit 1 rather than a Node stack trace exposing internal paths — the audience is a non-technical user already having a bad day. + - **Cloud-sync aware.** Desktop is preferred for findability, but an enterprise OneDrive-redirected Desktop would silently upload customer campaign data on write, contradicting the local-only guarantee. Synced folders are skipped, and the preview warns if only synced destinations exist. + - **Redaction is structural where the key is known.** The actions table matches on the parameter *name* rather than pattern-matching rendered text. An earlier regex-only approach that matched `secret:` / `authorization:` in prose shredded legitimate ad copy — "Secret: Summer Sale" became "Secret: `` Sale" — which is the opposite of what an evidence bundle is for. The remaining flat-text rule is `=`-only with a length floor; the JSON rule covers the colon form. + - **Table cells collapse whitespace.** A newline inside any tool-input value (creative titles and descriptions routinely have them) previously ended the Markdown row early and corrupted every row below it. + - **Fails loudly, never falsely.** An unreadable `--complaint-file` / `--title-file` exits 1 without writing, rather than emitting a bundle that tells support *"The user did not add a description"* about a user who did. `--out` refuses to overwrite an existing file without `--force`. Transcripts above 256 MB are refused with an explanation instead of an out-of-memory crash. + - Truncation closes any code fence or `
` block the cut left open. Multi-line titles are collapsed so the case's subject line survives. + - **Refuses to write inside a git working tree.** The bundle holds customer campaign data and this repo is public, so a bundle written into it is one `git add -A` from publication. SKILL.md already said not to, but an instruction to the model is not a control — this is enforced in the script, with `--allow-git` as a deliberate escape hatch. The default destination now also skips git work trees (a dotfiles-managed home directory is one) and falls back to the OS temp directory. + - **Object-valued parameters are serialized in the actions table** instead of flattening to `[object Object]` via `String()`. Realize targeting blocks are objects, so the previous rendering told PS nothing about exactly the payloads whose failures are hardest to diagnose. + - **Path comparison is separator-normalized.** `path.join` yields backslashes while a hand-written `--out` usually has forward slashes; a raw compare meant the OneDrive warning silently failed to fire on precisely the folder it exists to catch. + - Auto-generated filenames that collide (two runs in the same second) get a numeric suffix rather than erroring; an explicit `--out` collision still refuses without `--force`. Header rows relabelled "acted on" — the IDs come from request parameters, so a session that only browsed results correctly shows none. + - **Truncation is tiered by how diagnostic the output is.** Realize responses get 20,000 characters; other tool output stays at 2,000. A report CSV usually *is* the case — "the CPA here disagrees with the UI" is answered by the rows behind the number — and under the previous uniform cap only ~13 of 250 rows survived, so the disputed row was typically the one missing. Bulk output from unrelated tools stays capped so the bundle remains email-attachable. + - **The git refusal no longer names its own override.** `--allow-git` still exists for maintainers but appears in neither the error message nor `SKILL.md`: the model that hits the error reads that message as its next instruction, and the correct next step is a different path, not a bypass on the one control between customer data and a public repo. Documented in `CLAUDE.md` only. + - The skill now deletes the scratch `complaint.txt` / `title.txt` after writing the bundle. + - **Shipped as one skill, not a command plus a skill.** An earlier split (`commands/support.md` routing into a `support-bundle` skill) registered *both* as invocable, so users saw two near-identical entries — `/realize-plugin:support` and `/realize-plugin:support-bundle` — with no way to tell which was correct. The thin command added nothing the skill could not do, so it was removed and the skill renamed to `support`. One feature, one entry. Caught during first live testing. + - **Invocation is `/realize-plugin:support`, not `/support`.** Plugin components are namespaced by plugin name. Every doc, the guardrails escalation footer users actually see, and the test scenarios said the bare form, which fails with `Unknown command`. Corrected across `README.md`, `INSTALL.md`, `os/guardrails.md`, the skill, and the scenarios. Caught during first live testing. + +### Fixed +- **`INSTALL.md` / `README.md` local-dev instructions were unrunnable from inside the repo.** Both showed `claude --plugin-dir ./realize-claude-plugin` without a `cd`, so a user who had already cloned and entered the directory got a path resolving to `…/realize-claude-plugin/realize-claude-plugin`. That loads **nothing and prints no error** — the session starts normally and every plugin command returns `Unknown command`, which reads as a broken plugin rather than a bad path. Both now show `cd realize-claude-plugin` then `--plugin-dir .`, explain that the path is relative to the launch directory, and give the `claude --plugin-dir . plugin list` check (look for `realize-plugin@inline … ✔ loaded`) plus `plugin details` for the component inventory. +- **`skills/support/scripts/test-build-bundle.js`** — 36 dependency-free unit assertions over the redaction rules, table-cell safety, truncation, destination guards, and tool matching. Wired into `.github/workflows/validate.yml`. Written as a file rather than inline `node -e` on purpose: the rules under test are dense with backslashes, quotes, and dollar signs, and passing those through a shell produced two false review results — a "leak" and a "failure" that were both harness artifacts, not defects. + - **Session identification** uses `CLAUDE_CODE_SESSION_ID` rather than a newest-file heuristic; a project folder routinely holds several sessions, so the fallback is genuinely unreliable and the preview reports match confidence when it is used. + - Node-based (Node ships with Claude Code); no new dependency. Read scenarios `17` and `18` added to `tests/test-scenarios-read.md`. +- **`os/guardrails.md` — *Offer the support escalation path* section**, plus a carve-out under *Internal tools, skills, and infrastructure — never reference*. Without the carve-out the existing rules would have suppressed the feature: line-level bans on skill names, `@taboola.com` addresses, and local file paths each block a required part of the escalation message. The carve-out scopes the exception tightly to the `/support` command name, the `Support@taboola.com` alias, and the saved file's path — the ban on naming individual Taboola employees is untouched. The trigger rules separate hard triggers (a failed action, a reported mismatch against the Realize UI, an unresolved complaint) from judgment triggers (asking for a human, repeating a question, expressed doubt), cap the line at **once per conversation**, and forbid it on answers the user hasn't questioned — a support prompt on a working answer reads as low confidence. Adds a bullet to the silent self-check. - **`os/guardrails.md` — *Publisher and site framing — measurable performance, not character* section.** New runtime rule sitting between *Privacy and brand-safety language* and *Performance framing*. Establishes the approved phrasing pattern for site / publisher recommendations (*" sites with over "*) and ties it back to the existing frozen brand phrases (*Embedded publisher integrations*, *Code on page integrations*) so the rule reads as brand-voice extension, not a banned-word list. Adds a corresponding bullet to the silent self-check. Anchors to the brand-voice review by product leadership, June 2026. - **`scripts/brand-check.sh` defense-in-depth FAIL patterns**: *Publisher-character framing* and *Rescue / salvage framing*. Catches qualitative judgments applied to publisher inventory in static markdown — runtime is governed by the guardrail above, this is the build-time net. - **`os/guardrails.md` — *Reader framing — the operator, not a relay through them* section.** New runtime rule placed between *Publisher and site framing* and *Performance framing*. Encodes that the reader is the campaign operator (self-serve advertiser or the agency running campaigns on a brand's behalf) — not a Taboola Account Manager who will relay the answer to "the client". Approved patterns: second-person to the operator, neutral instructional voice, and possessive *"your client"* for the agency-on-behalf-of-a-brand case. Out of voice: relay posture (*"ask the advertiser to…"*, *"the client should…"*, *"set advertiser expectations"*, *"discuss with the client"*, *"how to communicate this to clients"*). Adds a corresponding bullet to the silent self-check. Anchors to the brand-voice review by product leadership, June 2026. diff --git a/CLAUDE.md b/CLAUDE.md index 85805e3..7de248f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,12 +26,18 @@ This is a thin Claude Code plugin that wraps the [Realize remote MCP](https://gi ├──► reports skill → 4 report tools (CSV output) ├──► optimize-campaign skill → diagnoses underperformance; hands write │ prescriptions to manage-campaigns - └──► manage-campaigns skill → 6 write tools: create_campaign, update_campaign, - create_native_item, update_native_item, - create_display_item, update_display_item. - Tiered preview-then-confirm with mandatory - ▶ WRITE TARGET account header. - UI fallback for delete/duplicate/bulk ops. + ├──► manage-campaigns skill → 6 write tools: create_campaign, update_campaign, + │ create_native_item, update_native_item, + │ create_display_item, update_display_item. + │ Tiered preview-then-confirm with mandatory + │ ▶ WRITE TARGET account header. + │ UI fallback for delete/duplicate/bulk ops. + │ + └──► support skill → NO MCP tools. Reads the local Claude Code + session transcript and renders one Markdown + file the user emails to Support@taboola.com. + Preview-then-confirm; writes locally only, + transmits nothing. Entry point: /support. │ ▼ ┌────────────────────────────────────────┐ @@ -47,6 +53,51 @@ This is a thin Claude Code plugin that wraps the [Realize remote MCP](https://gi ### No hooks This plugin does not use Claude Code hooks. The remote MCP handles token refresh at the transport layer, so adding hooks here would be overhead without benefit. +### The support bundle exports the transcript, never a summary +`support` is the one skill that touches no MCP tool. It reads the local Claude Code session transcript and renders it for Taboola Professional Services. + +Everything in the bundle above the transcript — the Summary, the diagnostic table, the failed-action list, the ordered action log — is extracted **mechanically by the script**, not written by the model. This is deliberate and worth preserving: the bundle exists precisely for cases where the plugin got something wrong, and a model-authored summary of its own mistake reproduces the mistake. Don't "improve" this by having the model narrate what went wrong. + +The model now authors **nothing** in the bundle. The case subject was the last model-written field and is now the user's complaint text (see *The Summary section is for PS's case Description* below). + +Three constraints that are easy to break by accident: + +- **Session identification uses `CLAUDE_CODE_SESSION_ID`.** A project folder normally holds several `.jsonl` sessions, so the newest-file fallback really can grab the wrong conversation. It exists only as a last resort and surfaces `confidence: guessed` when used — keep that surfaced. +- **Redaction strips credentials but keeps business IDs.** `account_id` / `campaign_id` / `item_id` are preserved on purpose; PS can't reproduce anything without them. Don't "harden" this by masking them. +- **No upload path, by design.** The script writes one local file and prints the path. Adding transmission would turn a local diagnostic into an outbound flow of customer campaign data — that's a privacy-review decision, not a refactor. +- **User text goes in by file, never as a shell argument.** `--complaint-file` / `--title-file` exist because a quoted shell argument silently rewrites the text: `$12.40` becomes `2.40`, `$500` becomes `00`, and backticks or `$(…)` execute. Currency is everywhere in this domain and users paste error text they didn't write. Collapsing this back to `--complaint "…"` for brevity reintroduces both the corruption and the injection. +- **Redact before shortening, and cover both value shapes.** Slicing first can cut a credential below the length the patterns match on. The actions table renders inputs as `key=value`, which a JSON-shaped pattern never matches — that gap leaked plaintext secrets in review. +- **Redaction targets credentials, not prose.** Where the parameter name is known, match on the key (`redactValue`); reserve pattern-matching for free text. An earlier regex that matched `secret:` / `authorization:` anywhere shredded legitimate ad copy — "Secret: Summer Sale" became "Secret: `` Sale". Over-redaction destroys the evidence the bundle exists to carry, which is worse than the leak it guards. +- **The git-work-tree refusal is a control, not a suggestion.** `findGitRoot` blocks writes inside any repo. Customer data committed to this public repo is the worst outcome this feature can produce, and it is one bad `--out` away. Don't relax it to a warning. + + A `--allow-git` override exists for maintainers who need a bundle inside a checkout deliberately. It is **intentionally absent from the refusal message and from SKILL.md**: the model that hits the error reads that message as its next instruction, and the correct next step is a different path, not a bypass. Don't "improve" the error by naming the flag. + +- **Realize tool results get a much larger truncation budget than other output** (`MAX_REALIZE_RESULT_CHARS` vs `MAX_RESULT_CHARS`). A report CSV usually *is* the case — "the CPA here disagrees with the UI" is answered by the rows behind the number. Under the old uniform 2,000-char cap only ~13 of 250 rows survived, so the disputed row was typically the one missing. Bulk output from other tools stays tightly capped so the bundle remains email-attachable. + +### The Summary section is for PS's case Description +Requested by the PS manager, and shaped by the constraint above. Case intake copies an email's **subject into the case Subject** and its **body into the case Description**, so the bundle now renders: + +- a copy-ready **subject** = the user's complaint, one line, capped at `MAX_SUBJECT_CHARS`, with the first account ID appended for triage; +- **§1 Summary** = `EMAIL_PROLOG` (*"This case has been created by the Realize Plugin…"*) followed by mechanically-extracted facts, which the user pastes as the email body. + +The tempting mistake is to satisfy "we want a summary" with a model-written narrative. That is the exact thing the section above forbids, so §1 carries only counts, tool names, and file paths. What makes it genuinely useful to PS is the attribution: **which Realize tools ran, which skills were invoked, and which knowledge files were read.** That separates *"the plugin read the right guidance and still got it wrong"* from *"the plugin never read it"* — different bugs with different fixes, previously indistinguishable from the outside. `knowledgeRef` is deliberately narrow (only `knowledge/`, `os/`, `agents/`, `skills/**/SKILL.md`, `skills/**/references/`); widening it to every file touched turns a signal into noise. + +It is also **anchored to the plugin root**, not pattern-matched anywhere in the path. Unanchored, a user's own `~/Documents/os/notes.md` was reported to PS as plugin guidance — worse than reporting nothing, because the section exists to answer "did it read the guidance?" and a false entry answers it wrongly. The root is derived from `__dirname`; `knowledgeRef` takes an injectable root so tests don't depend on the checkout location. + +Two more traps in this area, both found in review: + +- **The subject must not be squeezed out by its own suffix.** `account_id` is an opaque API string with no length bound. A `slice(0, MAX - suffix.length - 1)` goes negative on a long one and slices *from the end*, which replaced the entire complaint with a bare `…`. `MIN_SUBJECT_TEXT_CHARS` now guarantees the user's words win and the account suffix is dropped instead. +- **The complaint is redacted, not trusted.** It is the user's own prose, so it is never rewritten — but users paste error output they never read, and that string now travels into an email subject. `redact()` is safe to apply here precisely because its flat rule is `=`-only with a length floor, so sentences survive while `Bearer …` does not. + +§1 also states in the file that it is mechanical and points at the transcript. Keep that line — it is what stops a reader treating the bullet list as the plugin's testimony about itself. + +Run `node skills/support/scripts/test-build-bundle.js` after touching the script — CI runs it too. Add cases there rather than testing via inline `node -e`: the rules are dense with backslashes and dollar signs, and shell escaping produced two false results during review. + +### The guardrails carve-out for `/support` is load-bearing +`os/guardrails.md` bans surfacing skill names, `@taboola.com` addresses, and local file paths. The escalation message needs all three, so *Internal tools, skills, and infrastructure — never reference* carries an explicit carve-out. + +If you tighten those bans later, **re-check the carve-out** — without it the model silently stops offering `/support`, and the failure is invisible (a feature that quietly never fires, not an error). Scenario 18 in `tests/test-scenarios-read.md` is the regression test. + ### No direct curl / no API client code All Realize API access flows through MCP tools. Do not add Bash curl calls that hit Realize endpoints directly — that bypasses the MCP's rate limiting, auth handling, and safety guarantees. diff --git a/INSTALL.md b/INSTALL.md index 4f770cf..5811dd6 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -29,11 +29,28 @@ Use this to iterate on skills, run test scenarios, or work in a restricted/air-g ```bash git clone https://github.com/taboola/realize-claude-plugin -claude --plugin-dir ./realize-claude-plugin +cd realize-claude-plugin +claude --plugin-dir . ``` `--plugin-dir` loads the plugin (skills, agent, and MCP wiring) directly from the local directory — no marketplace required. +> **The path is relative to where you launch `claude`.** From *inside* the repo use `--plugin-dir .`; from the parent folder use `--plugin-dir ./realize-claude-plugin`. A path that doesn't exist loads **nothing and prints no error** — the session starts normally and every plugin command comes back as `Unknown command`. +> +> Check it loaded before doing anything else: +> +> ```bash +> claude --plugin-dir . plugin list +> ``` +> +> You want a `realize-plugin@inline … Status: ✔ loaded` entry. To see which skills and commands registered: +> +> ```bash +> claude --plugin-dir . plugin details realize-plugin@inline +> ``` + +**Invoking commands:** plugin components are namespaced by plugin name, so the support command is `/realize-plugin:support`, not `/support`. Typing `/` lists everything available in the session. + **Picking up code changes:** after `git pull`, run `/reload-plugins` inside the session to refresh without restarting the CLI. **Loading multiple plugins at once:** repeat the flag, e.g. `claude --plugin-dir ./realize-claude-plugin --plugin-dir ./other-plugin`. diff --git a/README.md b/README.md index 1a5f7ec..bd42f4f 100644 --- a/README.md +++ b/README.md @@ -32,9 +32,14 @@ Clone the repo and load it as a local plugin via `--plugin-dir`: ```bash git clone https://github.com/taboola/realize-claude-plugin -claude --plugin-dir ./realize-claude-plugin +cd realize-claude-plugin +claude --plugin-dir . ``` +> **The path is relative to where you launch `claude`.** From inside the repo use `.`; from the parent folder use `./realize-claude-plugin`. A path that doesn't exist loads **nothing and prints no error** — the session looks normal but every plugin command returns `Unknown command`. Verify with `claude --plugin-dir . plugin list` and look for `realize-plugin@inline … ✔ loaded`. + +Plugin components are namespaced by plugin name, so commands are invoked as `/realize-plugin:` (e.g. `/realize-plugin:support`), not bare `/support`. Typing `/` lists what's actually available in the session. + This loads the skills, agent, and MCP wiring directly from the repo without requiring a marketplace install. Use this for iterating on skills, testing scenarios, or running the plugin inside a restricted/air-gapped environment. On first tool call, Claude Code opens a browser for Taboola SSO login and returns you to the terminal once authenticated. @@ -67,6 +72,7 @@ This plugin wraps the remote [realize-mcp](https://github.com/taboola/realize-mc | [`reports`](skills/reports/SKILL.md) | Pull the four Realize performance reports and interpret the CSV output | | [`optimize-campaign`](skills/optimize-campaign/SKILL.md) | Diagnose underperforming campaigns against the toolkit's signal-quality thresholds (100+ clicks per item, daily spend ≥ 8× CPA goal, 7–14 day learning phase) and prescribe concrete actions (most now applied via `manage-campaigns`) | | [`manage-campaigns`](skills/manage-campaigns/SKILL.md) | Create and update campaigns and Native + Display items. Tiered preview-and-confirm pattern surfaces the target account on every write. Falls back to a UI reference for actions not supported here (delete, duplicate, bulk ops, Custom Rules, conversion-rule creation, CRM uploads) | +| [`support`](skills/support/SKILL.md) | Package the conversation into one file you can email to Taboola Support — see [`/realize-plugin:support`](#getting-help-with-a-problem) | **Start with a skill, not the MCP** — the skills carry the account-resolution rules, CSV conventions, optimization playbook, and write-preview gate that raw MCP calls bypass. The [`realize-analyst`](agents/realize-analyst.md) agent auto-routes natural-language questions to the right skill, or you can invoke one explicitly (e.g. `/realize-plugin:optimize-campaign`). The most common miss: treating "performance review" or "insights" as ad-hoc analysis when it belongs in `optimize-campaign`. @@ -132,10 +138,10 @@ The query window genuinely had no data, or you queried a campaign that didn't ru All three are **opaque identifiers** returned by the API. `account_id` is a string (e.g., `advertiser_12345_prod`) returned exclusively by `search_accounts`. `campaign_id` and `item_id` come from campaign/item tools. Pass them to follow-up calls exactly as received — don't reformat or coerce to numbers. **The plugin tried to create a campaign and failed.** -Claude routes write-intent requests to the `manage-campaigns` skill, which previews the resolved payload (and the target account) and asks for confirmation before submitting. Delete, duplicate, and bulk operations aren't supported here yet — those fall back to the Realize UI. If Claude attempted something unexpected, please [file an issue](https://github.com/taboola/realize-claude-plugin/issues) with the transcript. +Claude routes write-intent requests to the `manage-campaigns` skill, which previews the resolved payload (and the target account) and asks for confirmation before submitting. Delete, duplicate, and bulk operations aren't supported here yet — those fall back to the Realize UI. If Claude attempted something unexpected, run `/realize-plugin:support` to capture the transcript, then [file an issue](https://github.com/taboola/realize-claude-plugin/issues) with that file attached. **A write went to the wrong account.** -Every write preview must lead with `▶ WRITE TARGET: ()`. If you saw the wrong account in that header before approving, the issue is at the account-resolution step — re-run the `accounts` skill to confirm the right account is selected before retrying. If the header was missing entirely, that's a bug; please file an issue with the transcript. +Every write preview must lead with `▶ WRITE TARGET: ()`. If you saw the wrong account in that header before approving, the issue is at the account-resolution step — re-run the `accounts` skill to confirm the right account is selected before retrying. If the header was missing entirely, that's a bug — run `/realize-plugin:support` and send the resulting file to Support@taboola.com. **CSV output was truncated.** Very large result sets are auto-truncated server-side. Narrow the query (shorter date range, specific `campaign_id`, higher sort discrimination) and retry. @@ -144,6 +150,38 @@ Very large result sets are auto-truncated server-side. Narrow the query (shorter ## Support +### Getting help with a problem + +These conversations happen in your terminal, so Taboola Support can't see them. If the plugin gave you a wrong answer or you're stuck, run: + +``` +/realize-plugin:support +``` + +Optionally describe the problem in the same line: + +``` +/realize-plugin:support the CPA it reported doesn't match what I see in the Realize UI +``` + +You'll get a preview of what will be captured. Once you confirm, it saves a single Markdown file to your Desktop containing the full conversation — every question you asked, every action the plugin took, and every response it got back — plus a summary of which tools and guidance were used, and the account IDs involved. + +**Email it to [Support@taboola.com](mailto:Support@taboola.com).** The file opens with a "How to send this" section giving you the three pieces: + +| Email part | What to use | +|---|---| +| Subject | The copy-ready line at the top of the file (your own description of the problem) | +| Body | **Section 1. Summary** — copy it as-is | +| Attachment | The file itself, so Support gets the failed actions and full transcript | + +Subject and body are separated because Taboola's case system reads them into different fields — the subject becomes the case Subject, the body becomes the case Description. + +Why the whole transcript rather than a summary: if the plugin misread your question, a summary written by that same plugin would repeat the misreading. The raw exchange lets Support see what actually happened. + +**Your data:** nothing is sent anywhere automatically — the file is saved locally and emailing it is your decision. Credentials and tokens are stripped out. Account, campaign, and item IDs are deliberately kept, since Support can't reproduce an issue without them. The file also contains the campaign data shown in the session and your local folder paths; the preview tells you this before anything is written. + +### Bugs and feature requests + For product or security concerns, bug reports, and feature requests, open an issue at [github.com/taboola/realize-claude-plugin/issues](https://github.com/taboola/realize-claude-plugin/issues). diff --git a/agents/realize-analyst.md b/agents/realize-analyst.md index c0710c4..a4da44e 100644 --- a/agents/realize-analyst.md +++ b/agents/realize-analyst.md @@ -22,6 +22,8 @@ This plugin includes the **realize-toolkit**: a single system-prompt file (`os/g **For write-intent requests** (create/update a campaign or native item; pause/resume; budget or bid changes; targeting edits; creative swaps) → hand off to the `manage-campaigns` skill. It enforces the preview-then-confirm pattern with a mandatory `▶ WRITE TARGET: ()` header on every confirmation so the target account is never ambiguous. The per-write confirm gate is **not** bypassable, even when the user says *"don't ask before each one"* or *"just apply it"* — `manage-campaigns` refuses those framings. For requests with ambiguous scope (multiple possible targets), the skill confirms the exact target list before any preview. For delete/duplicate/bulk ops (no upstream MCP tool), the same skill falls back to a UI reference. +**For escalation to a human** (user wants a ticket, wants support, says an answer was wrong and a correction didn't resolve it, or hit an error they can't get past) → hand off to the `support` skill. It packages the conversation into one local file the user emails to Taboola Support. It previews before writing, writes nothing without confirmation, and transmits nothing. Do not summarize the problem for support yourself — the transcript is the evidence, and a summary written here would carry forward whatever this plugin got wrong. When to *offer* it unprompted is governed by *Offer the support escalation path* in `os/guardrails.md`. + **For MCP-driven questions** (account discovery, campaign inspection, reports) → use the skills below, applying `os/guardrails.md` to all output. --- diff --git a/os/guardrails.md b/os/guardrails.md index 8ee38cc..7d5b84c 100644 --- a/os/guardrails.md +++ b/os/guardrails.md @@ -185,6 +185,16 @@ Do not surface in user-facing output: The user's mental model: one assistant, doing things. Architecture is invisible. +#### Carve-out: the support escalation path + +The rules above ban skill names, `@taboola.com` addresses, and local file paths. The support escalation path is the **one exception**, because it is a user-facing product feature rather than internal architecture. When escalating (see *Offer the support escalation path* below), you may and should surface: + +- The **`/realize-plugin:support` command** by name (plugin components are namespaced; a bare `/support` is not a valid invocation and will fail for the user). It is a command the user types, not a skill they shouldn't know about — the same category as any documented feature. +- **`Support@taboola.com`** as the destination. The ban in the list above is on naming *individual* Taboola employees pulled from change logs or audit data; a published support alias is not that. +- **The saved file's path**, so the user can find it to attach. + +Nothing else opens up. Still never name the skill that builds the file, the MCP tools involved, or any repo/branch context. + ### Banned industry terms — use approved replacement | Do not say | Use instead | @@ -354,7 +364,7 @@ Users scan for the bottom line. Deliver the conclusion, not the workings. 1. **Bottom line first** (2-3 sentences max). The direct answer + most likely driver + 1-2 anchoring data points. 2. **Supporting detail** (only if needed). At most **3 bullets, one sentence each**. 3. **Closing question** — one open-ended question that doubles as the next step. -4. **Scope footer** in *italics*, last line. +4. **Scope footer** in *italics*, last line — or, when an escalation trigger fired, the support line instead (never both). See *Offer the support escalation path*. If the body (between bottom line and closing question) exceeds **6 lines or 3 one-sentence bullets**, cut. @@ -385,6 +395,34 @@ Shape: *"I can't [do the thing] — [one-sentence reason]. For [the legitimate p Never add a "Sources:" or "Tool calls:" footer enumerating the MCP tools that were used to produce the answer. The scope footer below (date range, account, filters, attribution model) is the only "sourcing" the user needs. Plugin internals — tool names, skill names, MCP routing — never appear in user output. Per the *Internal tools, skills, and infrastructure — never reference* rule above. +### Offer the support escalation path + +Users run this plugin in their own terminal, so Taboola support has **no visibility into these conversations**. If a user is stuck or doubts an answer and doesn't know `/realize-plugin:support` exists, the problem is invisible to anyone who could fix it. So the escalation line is offered on specific triggers — never on every answer. + +**Always offer it (hard triggers, no judgment required):** + +- A Realize action returned an error and the retry path is exhausted or unclear. +- The user reports a number here disagrees with the Realize UI. +- The user says an answer was wrong, and a corrected answer still doesn't satisfy them. + +**Offer it on judgment (implicit escalation intent):** + +- The user asks to speak to a person, open a ticket, or contact support. +- The user repeats the same question after an answer that didn't land. +- The user expresses clear frustration or doubt about the plugin's reliability. + +**Never offer it:** + +- On answers the user hasn't questioned. A support line on a working answer reads as low confidence in your own output. +- More than **once per conversation**, unless the user asks for it. Repeating it is nagging. +- Instead of actually solving the problem. Fix it first; escalation is the fallback, not the reflex. + +**Format** — one italic line, after the closing question, replacing the scope footer for that answer (never both): + +> *If this needs a human, run `/realize-plugin:support` and I'll package this conversation into a file you can email to Taboola Support.* + +Keep it to one sentence. Do not explain what the file contains, list its sections, or pitch it — the command explains itself when it runs. + ### Banned output patterns - Do not list every change-log entry — name only the 1-2 that matter. @@ -592,6 +630,7 @@ Before returning a response, verify: - [ ] Body ≤ 250 words for routine answers (write previews / multi-part diagnostics / structured tables exempted). - [ ] Refusals are short: one sentence + redirect. No enumeration of what could have been done, no internal-architecture walk-through, no hedging. - [ ] No "Sources:" or "Tool calls:" footer enumerating MCP tools. Scope footer (date, account, filters) is the only sourcing the user needs. +- [ ] Support escalation line appears only on a real trigger (failed action, UI mismatch, unresolved complaint, explicit ask for a human) — at most once per conversation, never on an unquestioned answer, and never alongside the scope footer. - [ ] If Target CPA was recommended, Maximize Conversions is also referenced. - [ ] If a write tool is about to be called, the `▶ WRITE TARGET` header is present, the preview was shown, and the user confirmed with an explicit Yes — per the **Write tool gate** section above. - [ ] Frozen phrases (Embedded publisher integrations, Proprietary Data Signals, Specialised performance AI, Code on page integrations, Performance outcomes at scale beyond search and social, Ads in Apple News and Stocks) appear unchanged. diff --git a/skills/support/SKILL.md b/skills/support/SKILL.md new file mode 100644 index 0000000..a32c9c1 --- /dev/null +++ b/skills/support/SKILL.md @@ -0,0 +1,137 @@ +--- +name: support +description: Package the current conversation into a single Markdown file the user can email to Taboola Support so Professional Services can debug what happened. Activates on /realize-plugin:support, and whenever the user wants to escalate to a human, report that the plugin gave a wrong or suspicious answer, or file a bug. Writes one local file after an explicit confirmation — never transmits anything. +allowed-tools: ["Read", "Bash", "AskUserQuestion"] +--- + +# Support Bundle + +Turns the current session into a support case file for Taboola Professional Services (PS). + +Users run the plugin in their own terminal, so PS has no visibility into these conversations. When someone says "the plugin gave me the wrong number", PS currently has only that sentence. This skill hands them the actual transcript instead. + +## When to use + +- The user runs `/realize-plugin:support` — plugin components are namespaced, so a bare `/support` is not a valid invocation. +- The user says the plugin gave wrong, stale, or suspicious information and wants it looked at. +- The user asks to talk to a person, open a ticket, or escalate. +- The user hit an error they can't get past and wants help. + +**Not** for answering product questions or diagnosing campaign performance — those belong to the normal skills. This skill only packages evidence. + +## The one rule that matters + +**The transcript is the deliverable.** Everything above it in the file is mechanically extracted from the transcript by the script — never written from your own recollection of the conversation. + +This is deliberate. If the plugin misread the user, a summary written by the plugin carries that same misreading into the bug report. Only the raw exchange is independent of the bug. So: do not editorialize, do not explain away what happened, and do not "clean up" the user's complaint. + +**Section 1 (Summary) is not an exception.** It exists because PS pastes it into the case Description, but every line in it is a count, a tool name, or a file path pulled from the log — never a narrative. It reports *which* Realize tools ran, *which* skills were invoked, and *which* knowledge files were read, so PS can tell "the plugin had the right guidance and still got it wrong" apart from "the plugin never read the guidance". Those are different bugs. Do not add a prose account of what went wrong to that section. + +**You write nothing into this bundle.** The case subject is now the user's own complaint text, so there is no field left for your judgment. + +## Workflow + +### Step 1 — Preview (never skip) + +```bash +node skills/support/scripts/build-bundle.js --preview +``` + +Writes nothing. Prints what would be captured: session ID, time window, turn counts, Realize actions attempted and how many failed, account/campaign IDs involved, local paths, and estimated file size. + +### Step 2 — Show the user what's about to leave their machine + +This file contains their real campaign data — budgets, spend, account IDs — plus their local folder paths. They must see that before a file exists. + +Present the preview in plain language: + +> This will save a file containing our whole conversation — **28 messages**, covering **account 1721090**, about **95 KB**. +> It includes the campaign data shown in this session and your local folder path. Credentials and tokens are removed automatically. +> Nothing is sent anywhere — the file is saved to your Desktop and you choose whether to email it. + +Then confirm with `AskUserQuestion`: create the file, or cancel. + +If the preview says match confidence is **guessed**, tell the user plainly: the exact session couldn't be identified and this may be the wrong conversation. Let them cancel. + +### Step 3 — The case subject writes itself + +**Do not compose a title.** The script uses the user's own complaint as the email subject, collapsed to one line, capped for a subject header, with the account ID appended for triage. Case intake copies that subject into the case Subject field, and the user's sentence is the most faithful description of their own problem available. + +`--title-file` still exists, but only as the fallback for a run where the user gave no description at all. If they described the problem, the description is the subject. + +### Step 4 — Write the file + +**Never pass the complaint or title as a quoted shell argument.** Write them to files first, then point the script at the files: + +1. `Write` the user's description verbatim to `complaint.txt` (use the session scratchpad or the system temp folder — never the plugin repo). +2. Run: + +```bash +node skills/support/scripts/build-bundle.js --write \ + --complaint-file /complaint.txt +``` + +This is not a style preference. The shell rewrites this text: `$12.40` arrives as `2.40`, `$500` arrives as `00`, and backticks or `$(…)` **execute**. Complaints in this domain are full of currency amounts, and users paste error text they didn't write. The whole point of the bundle is that the evidence reaches PS unaltered, so it must not pass through the shell. + +`--complaint` / `--title` still exist for trivial ASCII with no `$`, backticks, or quotes. When in doubt, use the file form. + +The complaint is the user's own description — use their words. If they ran the command with no description, ask one short question ("In one line, what went wrong?") and pass the answer through unedited. Don't write it for them. + +`--out ` overrides the destination. Default is Desktop, then Downloads, then home folder — skipping any that OneDrive has redirected to the cloud. + +### Step 5 — Tell them how to send it + +Three parts, because case intake maps each one to a different field. The file's own "How to send this to Support" section says the same thing — repeat it in chat so they don't have to open the file to know what to do: + +> Saved to `C:\Users\you\Desktop\realize-support-2026-08-09-...md` +> +> Email it to **Support@taboola.com**: +> - **Subject:** *the CPA in the report doesn't match the UI (account 1721090)* +> - **Body:** copy **Section 1. Summary** from the file +> - **Attach the file** so Support has the failed actions and full transcript + +Say plainly that nothing was sent automatically — sending is their decision. + +Then delete the scratch `complaint.txt` you created in Step 4. It holds the user's description of their problem and serves no purpose once the bundle exists. + +## What the bundle contains + +| Section | Purpose | +|---|---| +| How to send this to Support | Copy-ready subject, which section is the email body, and the reminder to attach the file | +| 1. Summary | **Pasted into the case Description.** Opens with the PS-requested prolog, then session facts, Realize tools used, skills invoked, knowledge files read, and any failures. Mechanically extracted — no narrative | +| 2. At a glance | Session ID, timestamps, turn counts, accounts/campaigns/items involved, actions attempted, **failed count**, plugin + Claude Code version, platform | +| 3. What the user reported | The `--complaint` text, verbatim. Also the source of the email subject | +| 4. Failed actions | Every failed Realize action with parameters sent and the error returned | +| 5. Realize actions attempted, in order | Ordered table of every Realize action and its result | +| 6. Full transcript | Every message, tool call, and result — with internal reasoning in collapsible blocks | + +## Privacy + +- **Nothing is transmitted.** The script writes one local file. There is no upload path, by design. +- **Credentials are stripped**: bearer tokens, `access_token`, `refresh_token`, `api_key`, `password`, `client_secret`, and JWTs. +- **Business IDs are kept.** `account_id`, `campaign_id`, and `item_id` are preserved on purpose — PS cannot reproduce an issue without them. +- **Local paths are included** as diagnostics, and the preview says so. +- Never write the bundle into the plugin repo or any git working tree — it contains customer data. Default destinations are outside the repo; keep it that way. + +## Gotchas + +- **Session identification** comes from `CLAUDE_CODE_SESSION_ID`. A project folder usually holds several sessions, so the "newest file" fallback is genuinely unreliable — that's why the preview reports match confidence, and why `guessed` must be surfaced to the user. +- **The current turn is mid-write.** The transcript is complete up to roughly the user's command message; the assistant reply being composed right now isn't in it yet. Harmless, but don't promise it captured "everything including this reply". +- **Large sessions get truncated.** Tool results cap at 2,000 characters, user messages at 4,000, internal reasoning at 1,200, and the bundle at 8 MB to stay email-attachable. Truncation is marked inline and never leaves a code fence or `
` block unclosed. If a specific large CSV matters to the case, tell the user to attach it separately. Transcripts above 256 MB are refused outright with an explanation rather than crashing. +- **An unreadable `--complaint-file` is fatal, and that's deliberate.** It exits 1 without writing. Continuing would produce a bundle stating *"The user did not add a description"* about a user who did describe the problem — a false statement to support is worse than no bundle. It would also silently drop the email subject, since the subject is derived from the complaint. If you see that error, check the path you wrote to. +- **The script refuses to overwrite.** If `--out` points at an existing file it exits 1 rather than clobbering it. Pass `--force` only when the user has asked to replace that specific file. (An auto-generated default name that collides simply gets a `-1` suffix — no error.) +- **The script refuses to write inside a git repository**, including this one, and exits 1. The bundle holds customer campaign data and this plugin's repo is public, so a bundle committed by accident is a data leak. If you see that error, choose a path outside the repo — the default destination already is one. +- **Unit tests:** `node skills/support/scripts/test-build-bundle.js` covers the redaction rules, table-cell safety, truncation, and destination guards. Run it after touching the script; CI runs it too. +- **No Realize actions in the session** is normal and not an error — it usually means the complaint is about the *content* of an answer, not a failed call. The bundle says so explicitly. +- **Node is required** and always present, since Claude Code itself runs on Node. Do not add a Python path. + +## Example prompts + +``` +/realize-plugin:support +/realize-plugin:support the CPA number doesn't match what I see in the UI +"This gave me the wrong budget — how do I report it?" +"Can I get a human to look at this?" +"I want to open a ticket about this." +``` diff --git a/skills/support/scripts/build-bundle.js b/skills/support/scripts/build-bundle.js new file mode 100644 index 0000000..d1cc626 --- /dev/null +++ b/skills/support/scripts/build-bundle.js @@ -0,0 +1,1087 @@ +#!/usr/bin/env node +'use strict'; + +/** + * Realize Plugin — support bundle builder. + * + * Reads the current Claude Code session transcript and renders a single + * human-readable Markdown file for the Taboola Professional Services team. + * + * The transcript is the payload: PS needs the exact prompt, the exact call, and + * the exact response. Everything above the transcript is mechanically derived + * from it — never model-interpreted — so a plugin that misunderstood the user + * cannot distort its own bug report. + * + * Usage: + * node build-bundle.js --preview + * node build-bundle.js --write --title-file --complaint-file + * [--out ] [--force] + * + * Pass user-authored text by file, not as a quoted argument: the shell rewrites + * it (`$12.40` becomes `2.40`) and `$(…)` executes. `--title` / `--complaint` + * remain for trivial ASCII only. + * + * Node ships with Claude Code, so it is always available. No dependencies. + */ + +const fs = require('fs'); +const path = require('path'); +const os = require('os'); + +const SUPPORT_EMAIL = 'Support@taboola.com'; + +// Prepended to the summary so the case opens with its own provenance. Requested +// by PS: a Description that starts mid-diagnostics reads like a forwarded stack +// trace, and the agent picking up the case has no idea what produced it. +const EMAIL_PROLOG = + 'This case has been created by the Realize Plugin, please find the interaction summary below.'; + +// Case intake puts the email subject into the case Subject field, which is a +// single short line. The user's own words are the subject (per PS), so they have +// to survive being cut to something a triage queue can read. +const MAX_SUBJECT_CHARS = 120; + +// Floor on how much of the user's own sentence must survive in the subject. If +// the account suffix would leave less than this, the suffix is dropped instead. +const MIN_SUBJECT_TEXT_CHARS = 40; + +const MAX_RESULT_CHARS = 2000; // per non-Realize tool result (bulk output) +const MAX_REALIZE_RESULT_CHARS = 20000; // per Realize result — this is the evidence +const MAX_THINKING_CHARS = 1200; // per thinking block +const MAX_BUNDLE_BYTES = 8 * 1024 * 1024; + +// --------------------------------------------------------------------------- +// args +// --------------------------------------------------------------------------- + +function parseArgs(argv) { + const out = { + preview: false, + write: false, + title: '', + complaint: '', + out: '', + force: false, + allowGit: false, + }; + for (let i = 0; i < argv.length; i++) { + const a = argv[i]; + if (a === '--preview') out.preview = true; + else if (a === '--write') out.write = true; + else if (a === '--force') out.force = true; + else if (a === '--allow-git') out.allowGit = true; + else if (a === '--title') out.title = argv[++i] || ''; + else if (a === '--complaint') out.complaint = argv[++i] || ''; + else if (a === '--title-file') out.title = readTextFile(argv[++i]); + else if (a === '--complaint-file') out.complaint = readTextFile(argv[++i]); + else if (a === '--out') out.out = argv[++i] || ''; + } + if (!out.preview && !out.write) out.preview = true; + // A title spanning multiple lines splits the header and gives PS a broken + // subject line. draftTitle() already collapses whitespace; do the same for + // titles that arrive from a file or argument — and give them the same + // redact-then-clamp treatment as the complaint, because the title becomes + // the email subject when no complaint was supplied, so it leaves the machine. + out.title = clampChars(redact(oneLine(out.title)), MAX_SUBJECT_CHARS); + return out; +} + +/** + * Read user-authored text from a file rather than an argv string. + * + * The complaint is the user's own words and routinely contains characters the + * shell rewrites — `$12.40` becomes `2.40` under double quotes, and `$(…)` or + * backticks execute. Since this text is the evidence PS reads, it has to arrive + * byte-for-byte. `--complaint` / `--title` remain for trivial ASCII, but the + * file variants are the supported path. + */ +function readTextFile(p) { + if (!p) throw new Error('--complaint-file / --title-file was given without a path.'); + try { + return fs.readFileSync(p, 'utf8').trim(); + } catch (err) { + // Fatal on purpose. Continuing would write a bundle whose "What the user + // reported" section reads "The user did not add a description" — an + // affirmative false statement to support about a user who did describe the + // problem. Losing the complaint is worse than failing loudly. + throw new Error(`could not read ${p} — ${err.message}`); + } +} + +// --------------------------------------------------------------------------- +// locating the transcript +// --------------------------------------------------------------------------- + +/** Claude Code encodes the project path by replacing every non-alphanumeric char with '-'. */ +function encodeProjectDir(cwd) { + return cwd.replace(/[^a-zA-Z0-9]/g, '-'); +} + +/** + * Resolve this session's transcript. + * + * CLAUDE_CODE_SESSION_ID is the authoritative signal. Falling back to + * "newest .jsonl" is a real coin flip — a project folder routinely holds + * several sessions — so that path is a last resort and is reported as such. + */ +function findTranscript() { + const projectsRoot = path.join(os.homedir(), '.claude', 'projects'); + if (!fs.existsSync(projectsRoot)) { + return { error: `No Claude Code project data found at ${projectsRoot}.` }; + } + + const sessionId = process.env.CLAUDE_CODE_SESSION_ID || ''; + const cwd = process.cwd(); + + if (sessionId) { + // Preferred: derive the folder from cwd. + const direct = path.join(projectsRoot, encodeProjectDir(cwd), `${sessionId}.jsonl`); + if (fs.existsSync(direct)) { + return { file: direct, sessionId, confidence: 'exact' }; + } + // The session may have started in a different cwd — scan every project folder. + for (const dir of safeReaddir(projectsRoot)) { + const candidate = path.join(projectsRoot, dir, `${sessionId}.jsonl`); + if (fs.existsSync(candidate)) { + return { file: candidate, sessionId, confidence: 'exact' }; + } + } + } + + // Last resort. + const dirPath = path.join(projectsRoot, encodeProjectDir(cwd)); + const candidates = safeReaddir(dirPath) + .filter((f) => f.endsWith('.jsonl')) + .map((f) => { + const full = path.join(dirPath, f); + return { full, mtime: fs.statSync(full).mtimeMs }; + }) + .sort((a, b) => b.mtime - a.mtime); + + if (!candidates.length) { + return { error: `No session transcript found for this project (looked in ${dirPath}).` }; + } + return { + file: candidates[0].full, + sessionId: path.basename(candidates[0].full, '.jsonl'), + confidence: 'guessed', + siblings: candidates.length, + }; +} + +function safeReaddir(p) { + try { + return fs.readdirSync(p); + } catch { + return []; + } +} + +// --------------------------------------------------------------------------- +// redaction +// --------------------------------------------------------------------------- + +/** + * Strip credentials only. Account, campaign, and item IDs are deliberately + * preserved — PS cannot reproduce the issue without them. + */ +const SECRET_KEYS = + 'password|passwd|client[-_]?secret|clientSecret|access[-_]?token|refresh[-_]?token|id[-_]?token|auth[-_]?token|api[-_]?key|apiKey|x-api-key|authorization|secret'; + +const REDACTIONS = [ + [/(Bearer\s+)[A-Za-z0-9._\-]{12,}/gi, '$1'], + // JSON form: "password": "value" + [new RegExp(`("(?:${SECRET_KEYS})"\\s*:\\s*")[^"]*(")`, 'gi'), '$1$2'], + // Flat `key=value` form, for text that is not JSON (error strings, logs). + // + // Deliberately `=` only, not `[:=]`. The colon form is already covered by the + // JSON rule above, and matching bare `secret:` / `authorization:` in prose + // shredded legitimate ad copy — "Secret: Summer Sale" became + // "Secret: Sale". Destroying creative text in a bundle whose whole + // purpose is evidence fidelity is worse than the leak it was guarding. + // + // The value must also be whitespace-free and reasonably long, so ordinary + // prose after an equals sign is left alone. + [ + new RegExp(`((?:${SECRET_KEYS})\\s*=\\s*)(?!', + ], + [/(eyJ[A-Za-z0-9_\-]{10,}\.[A-Za-z0-9_\-]{10,})\.[A-Za-z0-9_\-]+/g, ''], +]; + +/** Key names whose value is always a credential, matched structurally. */ +const SECRET_KEY_TEST = new RegExp(`^(?:${SECRET_KEYS})$`, 'i'); + +function redact(s) { + if (typeof s !== 'string') return s; + let out = s; + for (const [re, rep] of REDACTIONS) out = out.replace(re, rep); + return out; +} + +/** + * Redact a single parameter where the key is known. + * + * Preferred over running `redact()` across a rendered string: the key is + * structural data, so there is no need to guess at prose. Callers that have an + * object should use this. + */ +function redactValue(key, value) { + if (SECRET_KEY_TEST.test(String(key))) return ''; + return redact(String(value)); +} + +/** Collapse whitespace so a value can sit safely inside a Markdown table cell. */ +function oneLine(s) { + return String(s).replace(/\s+/g, ' ').trim(); +} + +function truncate(s, max) { + if (typeof s !== 'string') s = String(s); + if (s.length <= max) return s; + return `${s.slice(0, max)}\n… [truncated ${s.length - max} more characters]`; +} + +/** + * Cap by bytes, not characters, and leave valid Markdown behind. + * + * A character-based slice under a byte-based limit can overshoot on multi-byte + * content, and cutting inside a fenced block leaves every line below it + * rendering as code. + */ +function truncateToBytes(s, maxBytes, note) { + const buf = Buffer.from(s, 'utf8'); + if (buf.length <= maxBytes) return s; + + // Written as an escape, not the literal character: a raw replacement char in + // source is invisible in most editors and won't survive an encoding change. + let out = buf.subarray(0, maxBytes).toString('utf8').replace(new RegExp('\uFFFD+$'), ''); + + // Close anything the cut left open, or every line below it renders wrong. + if (((out.match(/^```/gm) || []).length) % 2 === 1) out += '\n```'; + const opened = (out.match(/
/g) || []).length; + const closed = (out.match(/<\/details>/g) || []).length; + for (let i = 0; i < opened - closed; i++) out += '\n
'; + + return `${out}\n\n${note}\n`; +} + +// --------------------------------------------------------------------------- +// parsing +// --------------------------------------------------------------------------- + +const MAX_TRANSCRIPT_BYTES = 256 * 1024 * 1024; + +function parseTranscript(file) { + // The whole transcript is read and rendered before the output cap applies, so + // guard the input too — otherwise a pathologically long session fails as an + // out-of-memory crash rather than a message anyone can act on. + const size = fs.statSync(file).size; + if (size > MAX_TRANSCRIPT_BYTES) { + throw new Error( + `this session's transcript is ${Math.round(size / 1024 / 1024)} MB, ` + + `above the ${MAX_TRANSCRIPT_BYTES / 1024 / 1024} MB limit. ` + + `Start a fresh session and reproduce the issue there, then run /support again.` + ); + } + + const raw = fs.readFileSync(file, 'utf8'); + const records = []; + for (const line of raw.split('\n')) { + if (!line.trim()) continue; + try { + records.push(JSON.parse(line)); + } catch { + /* tolerate partial trailing writes */ + } + } + return records; +} + +// Matches the plugin's own server (`mcp__realize-mcp__*`) and the claude.ai +// connector variant (`mcp__claude_ai_Realize_MCP__*`), which otherwise fell +// through to "other tools" and reported zero Realize activity. +const REALIZE_TOOL = /^mcp__(?:.*[-_])?realize[-_]?mcp__(.+)$/i; + +function analyze(records) { + const facts = { + sessionId: '', + claudeCodeVersion: '', + gitBranch: '', + cwd: '', + models: new Set(), + firstTs: '', + lastTs: '', + userTurns: 0, // messages the human actually typed + assistantTurns: 0, // replies containing prose, not tool-call-only records + toolResults: 0, // tool results, which the transcript also stores as "user" records + realizeCalls: [], // {name, input, ok, error, ts} + otherTools: new Map(), + // What guidance the plugin actually consulted before answering. PS needs + // this to separate two different bugs that look identical from outside: + // the plugin read the right guidance and still got it wrong, versus the + // plugin never read it. Extracted from tool calls, not self-reported. + knowledgeFiles: new Set(), + skillsUsed: new Set(), + accountIds: new Set(), + campaignIds: new Set(), + itemIds: new Set(), + firstUserText: '', + lastUserText: '', + }; + + // tool_use id -> call, so results can be matched back to their call + const callsById = new Map(); + + for (const r of records) { + if (r.sessionId && !facts.sessionId) facts.sessionId = r.sessionId; + if (r.version) facts.claudeCodeVersion = r.version; + if (r.gitBranch) facts.gitBranch = r.gitBranch; + if (r.cwd) facts.cwd = r.cwd; + if (r.timestamp) { + if (!facts.firstTs) facts.firstTs = r.timestamp; + facts.lastTs = r.timestamp; + } + if (r.message && r.message.model) facts.models.add(r.message.model); + + const content = r.message && r.message.content; + + // The transcript stores tool results as `type: "user"` records too, so a + // raw count of user records overstates what the human typed by ~10x. + // Count only records carrying actual text. + if (r.type === 'user') { + const text = extractText(content); + if (text) { + facts.userTurns++; + if (!facts.firstUserText) facts.firstUserText = text; + facts.lastUserText = text; + } + if (Array.isArray(content) && content.some((b) => b && b.type === 'tool_result')) { + facts.toolResults++; + } + } + // One reply spans many assistant records (one per tool call), so counting + // records — even records carrying text — overstates replies ~7x. + // `stop_reason: "end_turn"` marks a reply the user actually saw finish. + if (r.type === 'assistant' && r.message && r.message.stop_reason === 'end_turn') { + facts.assistantTurns++; + } + + if (!Array.isArray(content)) continue; + + for (const block of content) { + if (block.type === 'tool_use') { + const m = REALIZE_TOOL.exec(block.name || ''); + const input = block.input || {}; + collectIds(input, facts); + if (m) { + const call = { name: m[1], input, ok: null, error: '', ts: r.timestamp || '', id: block.id }; + facts.realizeCalls.push(call); + if (block.id) callsById.set(block.id, call); + } else if (block.name) { + // A recorded `Skill` call is already surfaced by name under "Skills + // invoked"; counting it here as well double-reports the same call. + if (!(block.name === 'Skill' && input.skill)) { + facts.otherTools.set(block.name, (facts.otherTools.get(block.name) || 0) + 1); + } + } + + if (block.name === 'Skill' && input.skill) facts.skillsUsed.add(String(input.skill)); + const fp = input.file_path || input.path || input.notebook_path; + const ref = typeof fp === 'string' ? knowledgeRef(fp) : ''; + if (ref) facts.knowledgeFiles.add(ref); + } + + if (block.type === 'tool_result') { + const call = block.tool_use_id ? callsById.get(block.tool_use_id) : null; + if (call) { + call.ok = !block.is_error; + if (block.is_error) call.error = truncate(redact(extractText(block.content)), 600); + } + } + } + } + + return facts; +} + +function collectIds(obj, facts) { + const json = JSON.stringify(obj || {}); + for (const m of json.matchAll(/"account_id"\s*:\s*"?([^",}]+)"?/g)) facts.accountIds.add(m[1]); + for (const m of json.matchAll(/"campaign_id"\s*:\s*"?([^",}]+)"?/g)) facts.campaignIds.add(m[1]); + for (const m of json.matchAll(/"item_id"\s*:\s*"?([^",}]+)"?/g)) facts.itemIds.add(m[1]); +} + +// The plugin's own guidance directories, matched against a path already made +// relative to the plugin root. Deliberately narrow: the question is "which +// guidance did it consult", not "every file it touched". A session that read the +// user's own spreadsheet has not consulted knowledge. +const KNOWLEDGE_REF = + /^(?:(?:knowledge|os|agents)\/[^/]+\.md|skills\/[^/]+\/(?:SKILL\.md|references\/[^/]+\.md))$/i; + +let cachedPluginRoot = null; + +/** This plugin's checkout root — `skills/support/scripts` is three levels down. */ +function pluginRoot() { + if (cachedPluginRoot === null) cachedPluginRoot = path.resolve(__dirname, '..', '..', '..'); + return cachedPluginRoot; +} + +/** + * Label a read file as a knowledge resource, or '' if it isn't one. + * + * Anchored to the plugin root rather than matched anywhere in the path. Without + * the anchor a user's own `~/Documents/os/notes.md` was reported to PS as plugin + * guidance, which is worse than reporting nothing: the section exists so PS can + * tell "read the right guidance and still got it wrong" from "never read it", + * and a false entry answers that question incorrectly. + * + * `root` is injectable for tests; production always uses the real checkout. + */ +function knowledgeRef(p, root) { + const baseFs = path.resolve(root === undefined ? pluginRoot() : String(root)).replace(/\\/g, '/'); + const fullFs = path.resolve(String(p)).replace(/\\/g, '/'); + + // Compared case-insensitively because Windows paths routinely differ only in + // case, but the label is sliced from the case-preserved path so `SKILL.md` + // keeps its name. + if (!fullFs.toLowerCase().startsWith(`${baseFs.toLowerCase()}/`)) return ''; + + const rel = fullFs.slice(baseFs.length + 1); + return KNOWLEDGE_REF.test(rel) ? rel : ''; +} + +function extractText(content) { + if (typeof content === 'string') return content; + if (!Array.isArray(content)) return ''; + return content + .filter((b) => b && (b.type === 'text' || typeof b.text === 'string')) + .map((b) => b.text || '') + .join('\n') + .trim(); +} + +// --------------------------------------------------------------------------- +// derived title +// --------------------------------------------------------------------------- + +/** + * A mechanical fallback title. The skill normally passes a better one via + * --title; this exists so the bundle is never untitled. + */ +function draftTitle(facts) { + const failed = facts.realizeCalls.filter((c) => c.ok === false); + if (failed.length) { + return `Realize Plugin — "${failed[0].name}" failed${ + facts.accountIds.size ? ` (account ${[...facts.accountIds][0]})` : '' + }`; + } + // firstUserText is raw chat text, and this drafted title becomes the email + // subject whenever the run had no complaint (emailSubject falls back here) — + // so it gets the same redact-first treatment as the complaint path, and the + // same backtick strip so the copy fence cannot close early. + const seed = redact(String(facts.firstUserText || '')) + .replace(/`/g, "'") + .replace(/\s+/g, ' ') + .trim(); + if (seed) { + return `Realize Plugin — ${seed.slice(0, 70)}${seed.length > 70 ? '…' : ''}`; + } + return 'Realize Plugin — support request'; +} + +/** + * The email subject, and therefore the case Subject field. + * + * PS asked for the user's own words here rather than a title written about them. + * That also removes the last model-authored field in the bundle, which is the + * direction the rest of this file already goes: the user reported the problem, + * so the user's sentence is the most faithful summary of it available. + * + * Collapsed to one line — a newline in a subject header truncates it at the + * break in most mail clients, silently dropping the rest. + */ +function emailSubject(complaint, facts) { + // Redacted first, then shortened — the ordering the rest of this file follows, + // because slicing can cut a credential below the length its pattern matches on. + // Users paste error output they did not author, and this string now travels + // into an email subject line, so a token in it would leave the machine. + // `redact` is safe on prose by construction: the flat rule is `=`-only with a + // length floor, so ordinary sentences are untouched. + // + // Backticks are stripped, not escaped: the subject is rendered inside a fenced + // block for copying, and a stray backtick there closes the fence early and + // swallows the rest of the file's formatting. + const text = redact(String(complaint || '')) + .replace(/`/g, "'") + .replace(/\s+/g, ' ') + .trim(); + if (!text) return draftTitle(facts); + + // Account ID earns its place in the subject: PS triages by account, and it is + // the one thing the user's sentence reliably omits. + const suffix = facts.accountIds.size ? ` (account ${[...facts.accountIds][0]})` : ''; + + // The user's words are the point of this subject, so they get the space first. + // account_id is an opaque API-supplied string with no length guarantee; when + // the suffix cannot fit, drop it rather than let it push the complaint out of + // the subject line describing that complaint. (A naive + // `slice(0, MAX - suffix.length - 1)` goes negative here and slices from the + // end, which silently replaced the entire complaint with an ellipsis.) + // Section 2 still lists every account acted on, so nothing is lost. + const room = MAX_SUBJECT_CHARS - suffix.length; + if (room < MIN_SUBJECT_TEXT_CHARS) return clampChars(text, MAX_SUBJECT_CHARS); + return `${clampChars(text, room)}${suffix}`; +} + +function clampChars(s, max) { + if (s.length <= max) return s; + return `${s.slice(0, Math.max(1, max - 1)).trimEnd()}…`; +} + +/** + * The Summary section, and the text the user pastes as the email body. + * + * Every line is derived from the session log. There is deliberately no written + * account of what went wrong: this bundle exists for the cases where the plugin + * misread the request, and a narrative composed by that same plugin reproduces + * the misreading in the one document meant to expose it. Counts, tool names and + * file paths cannot be wrong in that way, so those are what the summary carries. + */ +function renderSummary(facts) { + const failed = facts.realizeCalls.filter((c) => c.ok === false); + const succeeded = facts.realizeCalls.filter((c) => c.ok === true); + const actionNames = [...new Set(facts.realizeCalls.map((c) => c.name))]; + const L = []; + + L.push('## 1. Summary'); + L.push(''); + L.push(EMAIL_PROLOG); + L.push(''); + L.push(`- **Session:** \`${facts.sessionId || 'unknown'}\` — ${facts.firstTs || '?'} → ${facts.lastTs || '?'}`); + L.push(`- **Exchange:** ${facts.userTurns} message(s) from the user, ${facts.assistantTurns} repl(y/ies) from the plugin.`); + L.push(`- **Accounts acted on:** ${setOr(facts.accountIds, 'none')}`); + if (facts.campaignIds.size) L.push(`- **Campaigns acted on:** ${setOr(facts.campaignIds, 'none')}`); + if (facts.itemIds.size) L.push(`- **Items acted on:** ${setOr(facts.itemIds, 'none')}`); + L.push( + `- **Realize actions:** ${facts.realizeCalls.length} attempted — ` + + `${succeeded.length} succeeded, **${failed.length} failed**.` + ); + L.push( + `- **Realize tools used:** ${ + actionNames.length ? actionNames.map((n) => `\`${n}\``).join(', ') : 'none' + }` + ); + L.push( + `- **Skills invoked:** ${ + facts.skillsUsed.size ? [...facts.skillsUsed].map((s) => `\`${s}\``).join(', ') : 'none recorded' + }` + ); + L.push( + `- **Knowledge resources read:** ${ + facts.knowledgeFiles.size + ? [...facts.knowledgeFiles].map((k) => `\`${k}\``).join(', ') + : 'none recorded' + }` + ); + if (facts.otherTools.size) { + const others = [...facts.otherTools.entries()] + .sort((a, b) => b[1] - a[1]) + .slice(0, 8) + .map(([n, c]) => `\`${n}\`×${c}`) + .join(', '); + L.push(`- **Other tools used:** ${others}`); + } + + if (failed.length) { + L.push(''); + L.push('**Actions that failed:**'); + L.push(''); + for (const f of failed) { + const first = oneLine(f.error || '(no error text captured)'); + L.push(`- \`${f.name}\` — ${first.length > 160 ? `${first.slice(0, 160)}…` : first}`); + } + } + + L.push(''); + L.push( + '_Every line above is extracted mechanically from the session log; nothing here is ' + + 'the plugin\'s own account of what happened. That is deliberate — if the plugin ' + + 'misread the request, a summary it wrote would repeat the misreading. The verbatim ' + + 'exchange is in the attached file under "Full transcript"._' + ); + L.push(''); + return L.join('\n'); +} + +// --------------------------------------------------------------------------- +// rendering +// --------------------------------------------------------------------------- + +function pluginVersion() { + try { + const p = path.join(__dirname, '..', '..', '..', '.claude-plugin', 'plugin.json'); + return JSON.parse(fs.readFileSync(p, 'utf8')).version || 'unknown'; + } catch { + return 'unknown'; + } +} + +function renderHeader(facts, subject, complaint, loc) { + const failed = facts.realizeCalls.filter((c) => c.ok === false); + const L = []; + + L.push(`# Realize Plugin — Support Bundle`); + L.push(''); + L.push('## How to send this to Support'); + L.push(''); + L.push(`1. Start a new email to **${SUPPORT_EMAIL}**.`); + L.push('2. **Subject** — copy this line (it becomes the case Subject):'); + L.push(''); + L.push('```'); + L.push(subject); + L.push('```'); + L.push(''); + L.push('3. **Body** — copy all of **Section 1. Summary** below (it becomes the case Description).'); + L.push('4. **Attach this file**, so Support also has the failed actions and the full transcript.'); + L.push(''); + L.push('_Nothing has been sent anywhere. This file is on your machine and emailing it is your decision._'); + L.push(''); + L.push('---'); + L.push(''); + L.push(renderSummary(facts)); + L.push('## 2. At a glance'); + L.push(''); + L.push('| Field | Value |'); + L.push('|---|---|'); + L.push(`| Session ID | \`${facts.sessionId || 'unknown'}\` |`); + L.push(`| Conversation start | ${facts.firstTs || 'unknown'} |`); + L.push(`| Conversation end | ${facts.lastTs || 'unknown'} |`); + L.push(`| Messages from the user | ${facts.userTurns} |`); + L.push(`| Replies from the plugin | ${facts.assistantTurns} |`); + L.push(`| Tool results exchanged | ${facts.toolResults} |`); + // "acted on", not "involved": these come from the parameters of requests the + // plugin sent. A session that only browsed search results without acting on + // one correctly shows none here. + L.push(`| Accounts acted on | ${setOr(facts.accountIds, 'none')} |`); + L.push(`| Campaigns acted on | ${setOr(facts.campaignIds, 'none')} |`); + L.push(`| Items acted on | ${setOr(facts.itemIds, 'none')} |`); + L.push(`| Realize actions attempted | ${facts.realizeCalls.length} |`); + L.push(`| **Failed actions** | **${failed.length}** |`); + L.push(`| Plugin version | ${pluginVersion()} |`); + L.push(`| Claude Code version | ${facts.claudeCodeVersion || 'unknown'} |`); + L.push(`| Model(s) | ${setOr(facts.models, 'unknown')} |`); + L.push(`| Platform | ${os.platform()} ${os.release()} |`); + if (loc.confidence === 'guessed') { + L.push(`| ⚠ Transcript match | best-guess (session ID unavailable; ${loc.siblings} sessions in folder) |`); + } + L.push(''); + + L.push('## 3. What the user reported'); + L.push(''); + // Redacted, not paraphrased. The wording stays the user's own — `redact` only + // removes credential-shaped values, and users paste error output containing + // tokens they never looked at. + L.push(complaint ? redact(complaint) : '_The user did not add a description when generating this bundle._'); + L.push(''); + + L.push('## 4. Failed actions'); + L.push(''); + if (!failed.length) { + L.push('_No Realize action returned an error in this session. The reported problem is likely about the **content** of an answer rather than a failed call — see the transcript below._'); + } else { + for (const f of failed) { + L.push(`### \`${f.name}\` — failed${f.ts ? ` at ${f.ts}` : ''}`); + L.push(''); + L.push('Parameters sent:'); + L.push('```json'); + L.push(truncate(redact(JSON.stringify(f.input, null, 2)), 800)); + L.push('```'); + L.push('Error returned:'); + L.push('```'); + L.push(f.error || '(no error text captured)'); + L.push('```'); + L.push(''); + } + } + L.push(''); + + L.push('## 5. Realize actions attempted, in order'); + L.push(''); + if (!facts.realizeCalls.length) { + L.push('_No Realize actions were called in this session._'); + } else { + L.push('| # | Action | Result | Key parameters |'); + L.push('|---|---|---|---|'); + facts.realizeCalls.forEach((c, i) => { + const status = c.ok === false ? '❌ failed' : c.ok === true ? '✅ ok' : '– no result'; + // The key is known here, so redact structurally rather than pattern-match + // the rendered string. Collapse whitespace too: a newline inside a value + // (creative titles and descriptions routinely have them) would end the + // table row early and corrupt every row below it. + const keys = Object.entries(c.input || {}) + .slice(0, 4) + .map(([k, v]) => { + // Serialize objects rather than letting String() flatten them to + // "[object Object]". Realize targeting parameters are objects, and + // those are exactly the writes whose payload PS most needs to see — + // a truncated {"type":"EXCLUDE","value":[...]} still carries meaning. + const raw = v !== null && typeof v === 'object' ? JSON.stringify(v) : String(v); + const safe = oneLine(`${k}=${redactValue(k, raw)}`); + return safe.length > 48 ? `${safe.slice(0, 48)}…` : safe; + }) + .join(', '); + L.push(`| ${i + 1} | \`${c.name}\` | ${status} | ${escapePipes(keys) || '–'} |`); + }); + } + L.push(''); + L.push('---'); + L.push(''); + return L.join('\n'); +} + +function escapePipes(s) { + return String(s).replace(/\|/g, '\\|'); +} + +function setOr(set, fallback) { + return set && set.size ? [...set].join(', ') : `_${fallback}_`; +} + +function renderTranscript(records) { + const L = []; + L.push('## 6. Full transcript'); + L.push(''); + L.push('_Credentials are redacted. Long tool outputs are truncated — Realize responses far less aggressively than other output, since they are the evidence. Account, campaign, and item IDs are preserved so the issue can be reproduced._'); + L.push(''); + + // tool_use id -> whether it was a Realize call, so the result can be capped + // by how diagnostic it is rather than uniformly. + const realizeCallIds = new Set(); + for (const r of records) { + const c = r.message && r.message.content; + if (!Array.isArray(c)) continue; + for (const block of c) { + if (block.type === 'tool_use' && block.id && REALIZE_TOOL.test(block.name || '')) { + realizeCallIds.add(block.id); + } + } + } + + let n = 0; + for (const r of records) { + if (r.type !== 'user' && r.type !== 'assistant') continue; + const content = r.message && r.message.content; + const side = r.isSidechain ? ' _(background task)_' : ''; + const ts = r.timestamp ? ` · ${r.timestamp}` : ''; + + if (r.type === 'user') { + const text = extractText(content); + const results = Array.isArray(content) ? content.filter((b) => b.type === 'tool_result') : []; + if (text) { + n++; + L.push(`### 👤 User${side}${ts}`); + L.push(''); + L.push(quote(truncate(redact(text), 4000))); + L.push(''); + } + for (const res of results) { + // A report CSV is usually the whole case — "the CPA here disagrees with + // the UI" is answered by the rows that produced it. At the old uniform + // 2,000 chars only ~13 of 250 rows survived, so the disputed row was + // typically gone. Bulk output from other tools stays tightly capped. + const isRealize = res.tool_use_id && realizeCallIds.has(res.tool_use_id); + const cap = isRealize ? MAX_REALIZE_RESULT_CHARS : MAX_RESULT_CHARS; + const body = truncate(redact(extractText(res.content) || JSON.stringify(res.content || '')), cap); + L.push(`
${res.is_error ? '❌ ' : ''}Tool result${res.is_error ? ' (error)' : ''}`); + L.push(''); + L.push('```'); + L.push(body); + L.push('```'); + L.push(''); + L.push('
'); + L.push(''); + } + continue; + } + + if (!Array.isArray(content)) continue; + const header = `### 🤖 Assistant${side}${ts}`; + let wrote = false; + + for (const block of content) { + if (block.type === 'text' && block.text && block.text.trim()) { + if (!wrote) { L.push(header); L.push(''); wrote = true; } + L.push(truncate(redact(block.text), 6000)); + L.push(''); + } else if (block.type === 'thinking' && block.thinking) { + if (!wrote) { L.push(header); L.push(''); wrote = true; } + L.push('
Internal reasoning (why the plugin did this)'); + L.push(''); + L.push('```'); + L.push(truncate(redact(block.thinking), MAX_THINKING_CHARS)); + L.push('```'); + L.push(''); + L.push('
'); + L.push(''); + } else if (block.type === 'tool_use') { + if (!wrote) { L.push(header); L.push(''); wrote = true; } + const m = REALIZE_TOOL.exec(block.name || ''); + const label = m ? `Realize action: \`${m[1]}\`` : `Tool: \`${block.name}\``; + L.push(`**${label}**`); + L.push(''); + L.push('```json'); + L.push(truncate(redact(JSON.stringify(block.input || {}, null, 2)), 1200)); + L.push('```'); + L.push(''); + } + } + } + + if (!n) L.push('_No user messages were found in this transcript._'); + return L.join('\n'); +} + +function quote(s) { + return s.split('\n').map((l) => `> ${l}`).join('\n'); +} + +// --------------------------------------------------------------------------- +// output location +// --------------------------------------------------------------------------- + +/** + * Compare paths on a single canonical form. + * + * Windows mixes separators freely — `path.join` yields backslashes while a + * hand-written `--out` usually has forward slashes — and a raw string compare + * silently fails across the two, so the cloud-sync warning would not fire on + * exactly the folder it exists to catch. + */ +function normalizePath(p) { + return path.resolve(String(p)).replace(/\\/g, '/').toLowerCase(); +} + +/** Folders synced to a cloud provider, which would upload the bundle on write. */ +function cloudSyncedRoots() { + return [process.env.OneDrive, process.env.OneDriveCommercial, process.env.OneDriveConsumer] + .filter(Boolean) + .map(normalizePath); +} + +function isCloudSynced(p) { + const target = normalizePath(p); + return cloudSyncedRoots().some((root) => target === root || target.startsWith(`${root}/`)); +} + +/** + * Nearest enclosing git working tree, or null. + * + * The bundle holds customer campaign data. Landing it inside a repo — above all + * this one, which is public — puts it one `git add -A` away from being + * published. SKILL.md tells the model not to do that, but instructions are not + * a control; this is. + */ +function findGitRoot(startDir) { + let dir = path.resolve(startDir); + for (;;) { + if (fs.existsSync(path.join(dir, '.git'))) return dir; + const parent = path.dirname(dir); + if (parent === dir) return null; + dir = parent; + } +} + +/** + * Pick a destination. Desktop is the most findable place for a user who has to + * attach the file to an email, but it is often OneDrive-redirected on + * enterprise Windows (silently syncing customer data to the cloud), and a home + * directory is sometimes itself a git repo under a dotfiles setup. Prefer a + * folder that is neither; fall back to the OS temp directory, which is always + * safe on both counts, rather than knowingly choosing a bad one. + */ +function defaultOutPath(sessionId) { + const home = os.homedir(); + const stamp = new Date().toISOString().slice(0, 19).replace(/[:T]/g, '-'); + const name = `realize-support-${stamp}-${String(sessionId).slice(0, 8)}.md`; + + const candidates = ['Desktop', 'Downloads'] + .map((d) => path.join(home, d)) + .concat([home]) + .filter((d) => fs.existsSync(d)); + + const safe = candidates.find((d) => !isCloudSynced(d) && !findGitRoot(d)); + if (safe) return path.join(safe, name); + + const unsynced = candidates.find((d) => !findGitRoot(d)); + if (unsynced) return path.join(unsynced, name); + + return path.join(os.tmpdir(), name); +} + +/** Add a numeric suffix until the name is free. */ +function nextAvailablePath(p) { + const dir = path.dirname(p); + const ext = path.extname(p); + const base = path.basename(p, ext); + for (let i = 1; i < 1000; i++) { + const candidate = path.join(dir, `${base}-${i}${ext}`); + if (!fs.existsSync(candidate)) return candidate; + } + throw new Error(`could not find a free filename next to ${p}.`); +} + +// --------------------------------------------------------------------------- +// main +// --------------------------------------------------------------------------- + +function main() { + const args = parseArgs(process.argv.slice(2)); + const loc = findTranscript(); + + if (loc.error) { + console.error(`ERROR: ${loc.error}`); + process.exit(1); + } + + const records = parseTranscript(loc.file); + const facts = analyze(records); + const failed = facts.realizeCalls.filter((c) => c.ok === false); + + // PS asked for the user's own words as the subject, since case intake copies + // the subject into the case Subject field. When the user described the problem + // that description *is* the most faithful title available, and preferring it + // removes the last model-authored field in the bundle. `--title` stays as the + // fallback for a run with no description. + const subject = args.complaint + ? emailSubject(args.complaint, facts) + : args.title || draftTitle(facts); + + // Rendered once and reused, so --preview does not pay for a full render and + // then throw it away. + const body = truncateToBytes( + `${renderHeader(facts, subject, args.complaint, loc)}\n${renderTranscript(records)}\n`, + MAX_BUNDLE_BYTES, + `… [bundle truncated at ${Math.round(MAX_BUNDLE_BYTES / 1024 / 1024)} MB to stay email-attachable]` + ); + let outPath = args.out || defaultOutPath(facts.sessionId); + + // A bundle inside a repo is one `git add -A` from being published, and this + // plugin's own repo is public. Refuse an explicit --out that lands in a work + // tree; the auto-chosen default already avoids them. + const gitRoot = findGitRoot(path.dirname(path.resolve(outPath))); + if (gitRoot && !args.allowGit) { + throw new Error( + // Deliberately does not name the override flag. The model that hits this + // error reads the message as its next instruction, and what it should do + // is pick a different path — not reach for a bypass on the one control + // standing between customer data and a public repository. + `${outPath} is inside the git repository at ${gitRoot}. This bundle contains ` + + `customer campaign data and must never be committed. Write it somewhere ` + + `outside the repository — your Desktop is the default and is already safe.` + ); + } + + if (args.preview) { + console.log('PREVIEW — nothing has been written yet.'); + console.log(''); + console.log(`Transcript : ${loc.file}`); + console.log(`Match confidence: ${loc.confidence}${loc.confidence === 'guessed' ? ` (⚠ ${loc.siblings} sessions in folder)` : ''}`); + console.log(`Session ID : ${facts.sessionId}`); + console.log(`Window : ${facts.firstTs || '?'} → ${facts.lastTs || '?'}`); + console.log(`Messages : ${facts.userTurns} from you / ${facts.assistantTurns} replies`); + console.log(`Tool results : ${facts.toolResults}`); + console.log(`Realize actions : ${facts.realizeCalls.length} (${failed.length} failed)`); + console.log(`Accounts : ${facts.accountIds.size ? [...facts.accountIds].join(', ') : '(none seen)'}`); + console.log(`Campaigns : ${facts.campaignIds.size ? [...facts.campaignIds].join(', ') : '(none seen)'}`); + console.log(`Local paths : ${facts.cwd || '(none)'}`); + console.log(`Skills invoked : ${facts.skillsUsed.size ? [...facts.skillsUsed].join(', ') : '(none recorded)'}`); + console.log(`Knowledge read : ${facts.knowledgeFiles.size ? [...facts.knowledgeFiles].join(', ') : '(none recorded)'}`); + console.log(`Email subject : ${subject}`); + console.log(`Would write to : ${outPath}`); + console.log(`Size : ${Math.round(Buffer.byteLength(body, 'utf8') / 1024)} KB`); + if (isCloudSynced(outPath)) { + console.log(''); + console.log('WARNING: that folder is synced to OneDrive, so the file would upload to the'); + console.log(' cloud on write. Pass --out to choose a local-only folder.'); + } + return; + } + + // Never clobber an existing file without being told to. The default path is + // timestamped so collisions are unlikely, but --out is user-supplied and a + // silent overwrite destroys whatever was there. + if (fs.existsSync(outPath) && !args.force) { + if (args.out) { + // Explicit path: the caller named this file, so silently replacing it + // would destroy something they chose. + throw new Error( + `${outPath} already exists. Choose a different --out path, or pass --force to overwrite it.` + ); + } + // Auto-generated name (second-resolution timestamp): two runs in the same + // second are a collision, not a conflict. Step aside rather than fail. + outPath = nextAvailablePath(outPath); + } + + fs.mkdirSync(path.dirname(outPath), { recursive: true }); + fs.writeFileSync(outPath, body, 'utf8'); + + console.log('WROTE'); + console.log(`Path : ${outPath}`); + console.log(`Size : ${Math.round(Buffer.byteLength(body, 'utf8') / 1024)} KB`); + console.log(`Subject : ${subject}`); + console.log(`Send to : ${SUPPORT_EMAIL}`); + console.log('Email body : copy Section 1. Summary from the file'); + if (isCloudSynced(outPath)) { + console.log(''); + console.log('NOTE: this folder is synced to OneDrive, so a copy has gone to the cloud.'); + } +} + +// Exported so the redaction and parsing rules can be exercised directly by +// tests rather than re-typed into a test harness, where an escaping difference +// would prove nothing about this file. +module.exports = { + redact, + redactValue, + oneLine, + parseArgs, + analyze, + truncateToBytes, + REALIZE_TOOL, + draftTitle, + emailSubject, + renderSummary, + knowledgeRef, + EMAIL_PROLOG, + MAX_SUBJECT_CHARS, + isCloudSynced, + normalizePath, + findGitRoot, + defaultOutPath, + nextAvailablePath, + renderTranscript, + MAX_RESULT_CHARS, + MAX_REALIZE_RESULT_CHARS, +}; + +// Guarded with a conditional rather than a top-level `return`, which is valid +// in CommonJS only and would become a syntax error if this file is ever ESM. +if (require.main === module) { + runCli(); +} + +function runCli() { + try { + main(); + } catch (err) { + // The audience is a non-technical user already having a bad day; a raw + // stack trace exposing internal paths is not an acceptable failure mode. + console.error('ERROR: could not build the support bundle.'); + console.error(`Reason: ${err && err.message ? err.message : String(err)}`); + console.error(''); + console.error(`If this keeps happening, email ${SUPPORT_EMAIL} and mention this message.`); + process.exit(1); + } +} diff --git a/skills/support/scripts/test-build-bundle.js b/skills/support/scripts/test-build-bundle.js new file mode 100644 index 0000000..a000bd8 --- /dev/null +++ b/skills/support/scripts/test-build-bundle.js @@ -0,0 +1,417 @@ +#!/usr/bin/env node +'use strict'; + +/** + * Unit tests for the support-bundle builder. + * + * Run: node skills/support/scripts/test-build-bundle.js + * + * These exist as a file rather than an inline `node -e` because the rules under + * test are full of backslashes, quotes, and dollar signs. Passing those through + * a shell twice produced two false results during review — a "leak" and a + * "failure" that were both artifacts of the harness, not the code. A file has + * no shell layer. + * + * No dependencies, no framework: Node ships with Claude Code and that is the + * only thing this may assume. + */ + +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const b = require('./build-bundle.js'); + +let passed = 0; +const failures = []; + +function check(name, condition, detail) { + if (condition) { + passed++; + } else { + failures.push(detail ? `${name}\n ${detail}` : name); + } +} + +function eq(name, actual, expected) { + check(name, actual === expected, `expected ${JSON.stringify(expected)}, got ${JSON.stringify(actual)}`); +} + +// --------------------------------------------------------------------------- +// redaction — credentials out, business identifiers in +// --------------------------------------------------------------------------- + +eq('JSON password scrubbed', b.redact('{"password": "dbadmin"}'), '{"password": ""}'); +eq('JSON access_token scrubbed', b.redact('{"access_token": "ya29.abcdef"}'), '{"access_token": ""}'); +eq('flat password= scrubbed', b.redact('password=hunter2supersecret'), 'password='); +eq('flat client_secret= scrubbed', b.redact('client_secret=abcdef123456'), 'client_secret='); +check('bearer token scrubbed', b.redact('Bearer sk-ant-api03-XXXXXXXXXXXX').includes('')); +check('JWT scrubbed', b.redact('eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjMifQ.sig').includes('')); + +// Ad copy must survive. Over-redaction destroys the evidence the bundle exists +// to carry, which is worse than the leak it was guarding against. +eq('ad copy "Secret:" preserved', b.redact('title=Secret: Summer Sale'), 'title=Secret: Summer Sale'); +eq('ad copy "Authorization:" preserved', b.redact('title=Authorization: How To Apply'), 'title=Authorization: How To Apply'); +eq('ad copy "Password Manager" preserved', b.redact('name=Password Manager Review'), 'name=Password Manager Review'); + +// PS cannot reproduce an issue without these. +eq('account_id preserved', b.redactValue('account_id', 'advertiser_12345_prod'), 'advertiser_12345_prod'); +eq('campaign_id preserved', b.redactValue('campaign_id', '98765'), '98765'); +eq('structural redaction by key name', b.redactValue('password', 'anything at all'), ''); + +// --------------------------------------------------------------------------- +// table-cell safety +// --------------------------------------------------------------------------- + +check('newline collapsed for table cell', !/\n/.test(b.oneLine('a=Line one\nLine two'))); +check( + 'object param serialized, not [object Object]', + (() => { + const v = { type: 'EXCLUDE', value: ['p1', 'p2'] }; + const raw = v !== null && typeof v === 'object' ? JSON.stringify(v) : String(v); + const cell = b.oneLine(`publisher_targeting=${b.redactValue('publisher_targeting', raw)}`); + return cell.includes('EXCLUDE') && !cell.includes('[object Object]'); + })() +); + +// --------------------------------------------------------------------------- +// truncation leaves valid markdown +// --------------------------------------------------------------------------- + +{ + const src = `# H\n\n\`\`\`\n${'é'.repeat(400)}\nstill inside\n`; + const out = b.truncateToBytes(src, 200, '[cut]'); + check('byte cap respected', Buffer.byteLength(out, 'utf8') < 400); + check('code fence balanced', ((out.match(/^```/gm) || []).length) % 2 === 0); + check('no replacement char left', !new RegExp('�').test(out)); + eq('under-cap input untouched', b.truncateToBytes('short', 200, '[cut]'), 'short'); +} +{ + const src = '
x\n\n```\ndata here that goes on\n'; + const out = b.truncateToBytes(src, 40, '[cut]'); + check('
closed on truncation', (out.match(/<\/details>/g) || []).length >= 1); +} + +// --------------------------------------------------------------------------- +// destination safety +// --------------------------------------------------------------------------- + +{ + // Build the fake root with path.join so no backslash literals are involved. + const root = path.join('C:', 'Users', 'tester', 'OneDrive'); + const prev = process.env.OneDrive; + process.env.OneDrive = root; + + check('OneDrive path flagged (native separators)', b.isCloudSynced(path.join(root, 'Desktop', 'f.md'))); + check( + 'OneDrive path flagged (forward slashes)', + b.isCloudSynced(`${root.replace(/\\/g, '/')}/Desktop/f.md`) + ); + check('non-OneDrive path not flagged', !b.isCloudSynced(path.join('C:', 'Users', 'tester', 'Downloads', 'f.md'))); + check('no false prefix match on OneDriveX', !b.isCloudSynced(path.join('C:', 'Users', 'tester', 'OneDriveX', 'f.md'))); + + if (prev === undefined) delete process.env.OneDrive; + else process.env.OneDrive = prev; +} + +// Anchored to the checkout, not the caller's cwd, so the suite passes when run +// from anywhere. +check('this repo detected as a git work tree', b.findGitRoot(__dirname) !== null); +check('temp dir is not a git work tree', b.findGitRoot(os.tmpdir()) === null); +check('default destination is outside any git work tree', b.findGitRoot(path.dirname(b.defaultOutPath('abc12345'))) === null); + +{ + const p = path.join(os.tmpdir(), `collide-${process.pid}.md`); + fs.writeFileSync(p, 'x'); + const next = b.nextAvailablePath(p); + check('collision yields a free name', next !== p && !fs.existsSync(next)); + fs.unlinkSync(p); +} + +// --------------------------------------------------------------------------- +// result truncation is tiered by how diagnostic the output is +// --------------------------------------------------------------------------- + +{ + const bigCsv = `Records: 250 | Total: 1830\n${Array.from({ length: 400 }, (_, i) => `itm_${i},creative ${i},1234.56,987,0.44,37,33.37`).join('\n')}`; + + const records = [ + { + type: 'assistant', + message: { + content: [ + { type: 'tool_use', id: 'call_realize', name: 'mcp__realize-mcp__get_top_campaign_content_report', input: {} }, + { type: 'tool_use', id: 'call_bash', name: 'Bash', input: {} }, + ], + }, + }, + { + type: 'user', + message: { + content: [ + { type: 'tool_result', tool_use_id: 'call_realize', content: [{ type: 'text', text: bigCsv }] }, + { type: 'tool_result', tool_use_id: 'call_bash', content: [{ type: 'text', text: bigCsv }] }, + ], + }, + }, + ]; + + const out = b.renderTranscript(records); + const blocks = out.split('
').slice(1); + check('two tool results rendered', blocks.length === 2); + + const realizeRows = (blocks[0].match(/^itm_/gm) || []).length; + const bashRows = (blocks[1].match(/^itm_/gm) || []).length; + + check( + 'Realize result keeps far more rows than other output', + realizeRows > bashRows * 5, + `realize kept ${realizeRows} rows, bash kept ${bashRows}` + ); + check( + 'Realize result keeps a diagnostically useful number of rows', + realizeRows > 150, + `only ${realizeRows} rows survived` + ); + // ~42 chars per row against a 2,000-char cap lands near 47 rows. + check('non-Realize output stays tightly capped', bashRows < 60, `bash kept ${bashRows} rows`); + check('caps are actually different', b.MAX_REALIZE_RESULT_CHARS > b.MAX_RESULT_CHARS); +} + +// --------------------------------------------------------------------------- +// tool matching +// --------------------------------------------------------------------------- + +check('matches plugin MCP tools', b.REALIZE_TOOL.test('mcp__realize-mcp__search_accounts')); +check('matches claude.ai connector tools', b.REALIZE_TOOL.test('mcp__claude_ai_Realize_MCP__authenticate')); +check('does not match unrelated MCP tools', !b.REALIZE_TOOL.test('mcp__sage__assist')); +check('does not match plain tools', !b.REALIZE_TOOL.test('Bash')); + +// --------------------------------------------------------------------------- +// email subject — the user's words become the case Subject +// --------------------------------------------------------------------------- + +{ + const bare = { accountIds: new Set(), realizeCalls: [], firstUserText: '' }; + const withAcct = { accountIds: new Set(['advertiser_777_prod']), realizeCalls: [], firstUserText: '' }; + + eq( + 'complaint used verbatim as subject', + b.emailSubject('the CPA does not match the UI', bare), + 'the CPA does not match the UI' + ); + + // The whole reason user text is passed by file. If a subject ever shows + // "2.40" the shell got hold of it again. + check( + 'currency survives into the subject', + b.emailSubject('reported $12.40, UI shows $18.90', bare).includes('$12.40') + ); + + eq( + 'account id appended for triage', + b.emailSubject('spend is wrong', withAcct), + 'spend is wrong (account advertiser_777_prod)' + ); + + check( + 'newlines collapsed — a break truncates a real subject header', + !/\n/.test(b.emailSubject('line one\nline two', bare)) + ); + + check( + 'backticks stripped so the copy fence cannot close early', + !b.emailSubject('the `spend` number is off', bare).includes('`') + ); + + { + const long = 'x'.repeat(400); + const s = b.emailSubject(long, withAcct); + check('subject capped', s.length <= b.MAX_SUBJECT_CHARS, `got ${s.length}`); + check('capped subject still carries the account', s.includes('advertiser_777_prod')); + } + + // The complaint now travels into an email subject line, and users paste error + // output they never read. Redaction runs before shortening. + { + const leaky = 'I got this error: Bearer sk-ant-api03-AAAAAAAAAAAAAAAAAAAA'; + check('token in a complaint never reaches the subject', !b.emailSubject(leaky, bare).includes('sk-ant-api03')); + check('subject shows the redaction rather than dropping the line', b.emailSubject(leaky, bare).includes('')); + } + + // ...and the ad-copy rule still holds on the complaint: prose is not shredded. + eq( + 'complaint prose with a colon keyword survives', + b.emailSubject('the Secret: Summer Sale campaign is off', bare), + 'the Secret: Summer Sale campaign is off' + ); + + // Regression: account_id is an opaque API string with no length guarantee. A + // naive `slice(0, MAX - suffix.length - 1)` goes negative and slices from the + // end, which replaced the whole complaint with a bare "…" — deleting the user's + // own words from the subject describing their own problem. + { + const huge = { accountIds: new Set(['a'.repeat(140)]), realizeCalls: [], firstUserText: '' }; + const s = b.emailSubject('spend is wrong for yesterday', huge); + check('over-long account id does not blow the cap', s.length <= b.MAX_SUBJECT_CHARS, `got ${s.length}`); + check('over-long account id does not eat the complaint', s.startsWith('spend is wrong'), `got ${JSON.stringify(s)}`); + } + + // Review finding: with no complaint, emailSubject falls back to draftTitle, + // which built the subject from raw chat text — a pasted token walked straight + // into the email subject through the one door that skipped redaction. + { + const leakyChat = { + accountIds: new Set(), + realizeCalls: [], + firstUserText: 'help, I get Bearer sk-ant-api03-BBBBBBBBBBBBBBBBBBBB when loading', + }; + check('token in chat text never reaches a drafted title', !b.draftTitle(leakyChat).includes('sk-ant-api03')); + check('drafted title shows the redaction instead', b.draftTitle(leakyChat).includes('')); + check( + 'backticks stripped from drafted title like the complaint path', + !b.draftTitle({ accountIds: new Set(), realizeCalls: [], firstUserText: 'the `spend` is off' }).includes('`') + ); + } + + // Review finding: --title / --title-file got oneLine() but neither the + // MAX_SUBJECT_CHARS cap nor redaction — both only applied inside + // emailSubject(), which the title path bypasses. + { + const long = b.parseArgs(['--title', 'y'.repeat(400)]); + check('cli title clamped to the subject cap', long.title.length <= b.MAX_SUBJECT_CHARS, `got ${long.title.length}`); + const leaky = b.parseArgs(['--title', 'crash log: api_key=sk-ant-api03-CCCCCCCCCCCCCCCCCCCC end']); + check('token in a cli title never survives parsing', !leaky.title.includes('sk-ant-api03')); + } + + // No complaint: must still be titled, never blank. + check( + 'falls back to a drafted title with no complaint', + b.emailSubject('', { accountIds: new Set(), realizeCalls: [], firstUserText: 'show me spend' }).length > 0 + ); +} + +// --------------------------------------------------------------------------- +// knowledge / skill attribution — which guidance was actually consulted +// --------------------------------------------------------------------------- + +// The real checkout, so the default (root-less) call is exercised too. +const ROOT = path.resolve(__dirname, '..', '..', '..'); +const FAKE = path.join('C:', 'somewhere', 'plugin'); + +eq('knowledge file recognized', b.knowledgeRef(path.join(FAKE, 'knowledge', 'bidding.md'), FAKE), 'knowledge/bidding.md'); +eq('guardrails recognized', b.knowledgeRef(path.join(FAKE, 'os', 'guardrails.md'), FAKE), 'os/guardrails.md'); +eq('agent file recognized', b.knowledgeRef(path.join(FAKE, 'agents', 'realize-analyst.md'), FAKE), 'agents/realize-analyst.md'); +eq('SKILL.md recognized', b.knowledgeRef(path.join(FAKE, 'skills', 'reports', 'SKILL.md'), FAKE), 'skills/reports/SKILL.md'); +eq( + 'skill reference recognized', + b.knowledgeRef(path.join(FAKE, 'skills', 'reports', 'references', 'csv-examples.md'), FAKE), + 'skills/reports/references/csv-examples.md' +); +eq('SKILL.md keeps its casing', b.knowledgeRef(path.join(FAKE, 'skills', 'x', 'SKILL.md'), FAKE), 'skills/x/SKILL.md'); +eq('unrelated file in root ignored', b.knowledgeRef(path.join(FAKE, 'budget.md'), FAKE), ''); +eq('non-markdown ignored', b.knowledgeRef(path.join(FAKE, 'knowledge', 'notes.txt'), FAKE), ''); + +// The false positive the root anchor exists to kill: a user's own directory that +// happens to be named like one of ours must not be reported to PS as guidance. +eq( + 'knowledge-shaped path outside the plugin root ignored', + b.knowledgeRef(path.join('C:', 'Users', 'me', 'Documents', 'os', 'notes.md'), FAKE), + '' +); +eq( + 'sibling directory sharing a prefix ignored', + b.knowledgeRef(path.join('C:', 'somewhere', 'pluginX', 'knowledge', 'bidding.md'), FAKE), + '' +); + +// Default root: resolves against the actual checkout with no argument. +eq( + 'real plugin file resolves with no root argument', + b.knowledgeRef(path.join(__dirname, '..', 'SKILL.md')), + 'skills/support/SKILL.md' +); +check('file outside the real checkout ignored', b.knowledgeRef(path.join(os.tmpdir(), 'knowledge', 'x.md')) === ''); + +// --------------------------------------------------------------------------- +// Summary section — mechanical, and says so +// --------------------------------------------------------------------------- + +{ + const records = [ + { + type: 'assistant', + sessionId: 'sess-1', + timestamp: '2026-08-10T10:00:00Z', + message: { + stop_reason: 'end_turn', + content: [ + { type: 'text', text: 'working on it' }, + { type: 'tool_use', id: 't1', name: 'Skill', input: { skill: 'realize-plugin:reports' } }, + { type: 'tool_use', id: 't2', name: 'Read', input: { file_path: path.join(ROOT, 'knowledge', 'bidding.md') } }, + { type: 'tool_use', id: 't2b', name: 'Read', input: { file_path: path.join('C:', 'Users', 'me', 'os', 'private.md') } }, + { type: 'tool_use', id: 't3', name: 'mcp__realize-mcp__get_campaign_breakdown_report', input: { account_id: 'acct_9' } }, + { type: 'tool_use', id: 't4', name: 'mcp__realize-mcp__search_accounts', input: { query: '123' } }, + ], + }, + }, + { + type: 'user', + message: { + content: [ + { type: 'tool_result', tool_use_id: 't3', is_error: true, content: [{ type: 'text', text: '403 Forbidden — not permitted' }] }, + { type: 'tool_result', tool_use_id: 't4', content: [{ type: 'text', text: 'ok' }] }, + ], + }, + }, + ]; + + const facts = b.analyze(records); + eq('skill invocation captured', [...facts.skillsUsed].join(','), 'realize-plugin:reports'); + eq('knowledge read captured', [...facts.knowledgeFiles].join(','), 'knowledge/bidding.md'); + eq('two realize calls seen', facts.realizeCalls.length, 2); + check('recorded Skill call not double-counted in otherTools', !facts.otherTools.has('Skill')); + eq('Read still counted in otherTools', facts.otherTools.get('Read'), 2); + + const s = b.renderSummary(facts); + check('summary opens with the PS prolog', s.includes(b.EMAIL_PROLOG)); + check('summary names the realize tools used', s.includes('get_campaign_breakdown_report') && s.includes('search_accounts')); + check('summary names the skill used', s.includes('realize-plugin:reports')); + check('summary names the knowledge read', s.includes('knowledge/bidding.md')); + check('summary reports the account', s.includes('acct_9')); + check('summary reports the failure count', /1 failed/.test(s)); + check('summary lists the failing action and its error', s.includes('403 Forbidden')); + check('summary discloses that it is mechanical', /extracted mechanically/.test(s)); + check('summary points at the transcript', /Full transcript/.test(s)); + + // Structural rather than keyword-based: prove the section contains no free + // prose line at all, instead of hoping a blacklist anticipates the phrasing a + // future edit would use. + { + const prose = s.split('\n').filter((ln) => { + const t = ln.trim(); + if (!t) return false; // blank + if (t.startsWith('## ')) return false; // heading + if (t.startsWith('- ')) return false; // fact bullet + if (/^\*\*.+:\*\*$/.test(t)) return false; // bold label + if (t === b.EMAIL_PROLOG) return false; // PS-requested prolog + if (t.startsWith('_') && t.endsWith('_')) return false; // mechanical disclaimer + return true; + }); + check('summary contains no free prose lines', prose.length === 0, `unexpected: ${JSON.stringify(prose)}`); + } + + // Empty session must still render a valid summary rather than throwing. + const empty = b.analyze([]); + const es = b.renderSummary(empty); + check('empty session still summarizes', es.includes(b.EMAIL_PROLOG) && /none/.test(es)); +} + +// --------------------------------------------------------------------------- + +console.log(`${passed} passed, ${failures.length} failed`); +if (failures.length) { + console.error(''); + for (const f of failures) console.error(` FAIL: ${f}`); + process.exit(1); +} diff --git a/tests/test-scenarios-read.md b/tests/test-scenarios-read.md index 8cbfdd6..106c0e5 100644 --- a/tests/test-scenarios-read.md +++ b/tests/test-scenarios-read.md @@ -260,3 +260,60 @@ Scenarios are roughly ordered from simplest to most involved; later ones depend 3. Returns the enum values verbatim. **Pass criteria:** No `account_id` is sent; values are presented as the exact enum strings the user would paste into a campaign setup. + +--- + +## 17. Support bundle — capture a session for Professional Services + +**Setup:** run at least one real Realize action first (e.g. scenario 5) so the bundle has actions to report. + +**User prompt:** +> `/realize-plugin:support the CPA showed $12.40 but the Realize UI shows $18.90 for the same range` + +The currency amounts are the point of this scenario, not decoration — see the pass criteria. + +**Expected behavior:** +1. The `support` skill activates. +2. Runs the builder with `--preview`. **Nothing is written yet.** +3. Reports back in plain language: turn count, approximate file size, the account IDs involved, and that local folder paths are included. States that credentials are stripped and nothing is transmitted. +4. Asks for explicit confirmation via `AskUserQuestion`. +5. On confirm, writes the file and reports the path, the suggested case title, and `Support@taboola.com` as the destination. + +**Pass criteria:** +- No file exists on disk before the user confirms. +- The text after the command appears **verbatim** in the bundle's "What the user reported" section — not paraphrased, not "improved". +- **`$12.40` and `$18.90` appear intact.** If they render as `2.40` / `8.90`, the complaint was passed as a quoted shell argument and the shell ate `$1` — the skill must write it to a file and use `--complaint-file`. Regression test for silent corruption of the exact figures the case is about. +- The counts are the conversation, not the log: "Messages from the user" matches what the human actually typed (a 7-message exchange reads 7, not 60+); tool results are counted separately. +- **Section 5's table is not broken.** Every row renders as one row. A creative title or description containing a newline must not split a row — that corrupts the table from that point down. +- **The email subject is the user's own sentence**, not a title written about it — `$12.40` / `$18.90` intact, one line, account ID appended. It appears in a copy-ready fenced block under "How to send this to Support". +- **Section 1 (Summary) opens with** *"This case has been created by the Realize Plugin…"* and lists the Realize tools called, skills invoked, and knowledge files read (e.g. `os/guardrails.md`, `skills/reports/SKILL.md`). +- **Section 1 contains no narrative** — no sentence describing what the plugin thought, intended, or got wrong. Counts, tool names, and file paths only, closing with the line stating it is mechanically extracted. A prose account here is a fail even if accurate: it's the one thing the bundle must not carry. +- **Ad copy survives redaction.** If the session touched a creative headed *"Secret: Summer Sale"* or similar, it appears intact. Redaction targets credentials, and mangling creative text defeats the purpose of the bundle. +- Re-running the command does not overwrite an earlier bundle; the second run either picks a new timestamped name or refuses. +- The suggested title names the symptom and includes the account ID; it does **not** assert a cause. +- Section 5 lists every Realize action attempted in order with its result. +- `grep -iE '"(access_token|refresh_token|password|client_secret)"\s*:\s*"[^<]' ` returns nothing. +- `account_id` / `campaign_id` values **are** present (PS needs them to reproduce). +- The file is written outside the plugin repo — it contains customer data and must never land in a git working tree. +- The preview reports match confidence `exact`. If it reports `guessed`, the skill must say so and offer to cancel. + +--- + +## 18. Support escalation is offered on a failed action — but not otherwise + +**Part A — offered.** Trigger a failing action (scenario 10's invalid `account_id` works). + +**Expected behavior:** after explaining the failure and the retry path, the answer ends with a single italic line offering `/realize-plugin:support`. + +**Pass criteria:** +- Exactly one sentence, offered **once**. Not repeated on the next answer. +- The scope footer is not also present on that answer — one or the other, never both. +- It does not describe what the bundle contains or list its sections. + +**Part B — not offered.** Run a normal successful report (scenario 5) with no complaint. + +**Pass criteria:** no support line appears. Offering escalation on an answer the user hasn't questioned reads as low confidence and is a fail. + +**Part C — implicit intent.** After any answer, say *"can I talk to a real person about this?"* + +**Pass criteria:** routes to `support` rather than replying that it can't connect the user to a human.