Skip to content

Add --context-lens flag#48

Open
knocte wants to merge 19 commits into
masterfrom
wip/context-lens-continued
Open

Add --context-lens flag#48
knocte wants to merge 19 commits into
masterfrom
wip/context-lens-continued

Conversation

@knocte

@knocte knocte commented May 27, 2026

Copy link
Copy Markdown
Contributor

Supersedes #41

Because after I tried last state of that PR, I encountered the following problems:

  • default to building context-lens with npm, but it should only do that if pnpm is not available (fixed here in this PR)
  • crash at start (fixed here in this PR):
Launching Pi using context-lens warapper with aidev user (sudo is required to impersonate 'aidev' user)...
The context-lens UI is available at http://localhost:4041/
node:internal/modules/cjs/loader:1478
throw err;
^
Error: Cannot find module '/Users/aidev/context-lens/dist/cli.js'
at Module._resolveFilename (node:internal/modules/cjs/loader:1475:15)
at wrapResolveFilename (node:internal/modules/cjs/loader:1048:27)
at defaultResolveImplForCJSLoading (node:internal/modules/cjs/loader:1072:10)
at resolveForCJSWithHooks (node:internal/modules/cjs/loader:1093:12)
at Module._load (node:internal/modules/cjs/loader:1261:25)
at wrapModuleLoad (node:internal/modules/cjs/loader:255:19)
at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:154:5)
at node:internal/main/run_main_module:33:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
  • issue at start (fixed here in this PR), because mitmproxy couldn't be executed by aidev:
Launching Pi using context-lens warapper with aidev user (sudo is required to impersonate 'aidev' user)...
The context-lens UI is available at http://localhost:4041/
🔍 Starting Context Lens proxy and analysis server...
📡 Captures → http://localhost:4041/api/ingest
@contextio/proxy running on http://127.0.0.1:4040/
🔤 Tokenizer initialized (cl100k_base + o200k_base)
🌐 Context Lens Analysis running on http://127.0.0.1:4041/
🔒 Starting mitmproxy (forward proxy for HTTPS interception)...
Failed to start mitmproxy: spawn mitmdump EACCES
Install it: pipx install mitmproxy

(Note, the solution was to run CL via current user while keep running Pi with aidev via spi.)

  • context-lens dashboard issue at start (fixed here in this PR):
Context Lens UI bundle not found
Build the UI and restart the server:

cd ui && pnpm install && pnpm run build
  • context-lens and Pi finally launching fine, but no traffic is captured (fixed here in this PR by copy-pasting some env vars in launcher script):
  735     apiKeyExport: Option<{ name: string; value: string }> = Nothing
  736 ): Promise<void> {
  737     const exportPrefix = getExportPrefix(apiKeyExport);
  738     const command = `
- 739 FULL_SUDO_CMD="${exportPrefix}export npm_config_prefix=$AGENT_USER_HOME/.npm-global && umask ${DEFAULT_UMASK} && cd $CURRENT_DIR && ${piBinaryPath} $@"
+ 739 PROXY_ENV=""
+ 740 [ -n "$https_proxy" ] && PROXY_ENV="$PROXY_ENV export https_proxy=\\"$https_proxy\\" &&"
+ 741 [ -n "$HTTPS_PROXY" ] && PROXY_ENV="$PROXY_ENV export HTTPS_PROXY=\\"$HTTPS_PROXY\\" &&"
+ 742 [ -n "$NPM_CONFIG_HTTPS_PROXY" ] && PROXY_ENV="$PROXY_ENV export NPM_CONFIG_HTTPS_PROXY=\\"$NPM_CONFIG_HTTPS_PROXY\\" &&"
+ 743 [ -n "$WSS_PROXY" ] && PROXY_ENV="$PROXY_ENV export WSS_PROXY=\\"$WSS_PROXY\\" &&"
+ 744 [ -n "$NODE_USE_ENV_PROXY" ] && PROXY_ENV="$PROXY_ENV export NODE_USE_ENV_PROXY=\\"$NODE_USE_ENV_PROXY\\" &&"
+ 745 [ -n "$SSL_CERT_FILE" ] && PROXY_ENV="$PROXY_ENV export SSL_CERT_FILE=\\"$SSL_CERT_FILE\\" &&"
+ 746 [ -n "$NODE_EXTRA_CA_CERTS" ] && PROXY_ENV="$PROXY_ENV export NODE_EXTRA_CA_CERTS=\\"$NODE_EXTRA_CA_CERTS\\" &&"
+ 747 [ -n "$REQUESTS_CA_BUNDLE" ] && PROXY_ENV="$PROXY_ENV export REQUESTS_CA_BUNDLE=\\"$REQUESTS_CA_BUNDLE\\" &&"
+ 748 
+ 749 FULL_SUDO_CMD="\${exportPrefix}\$PROXY_ENV export npm_config_prefix=$AGENT_USER_HOME/.npm-global && umask ${DEFAULT_UMASK} && cd $CURRENT_DIR && ${piBinaryPath} \$@"
  740 echo "Launching Pi with ${AGENT_USER} user (sudo is required to impersonate '${AGENT_USER}' user)..."
  741 exec sudo -i -u ${AGENT_USER} bash -c "$FULL_SUDO_CMD"`;
  742     await createLauncherScript(command, LAUNCHER_SCRIPT_FILENAME, apiKeyExport);
  743 }
  • Last but not least: increased the 15s context-lens timeout from 45s (with a 2nd patch).

@knocte

knocte commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

@knocte

knocte commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

patches: remove patch that's no longer needed

No no no, this patch was created in this PR, so this commit has to be squashed.

@webwarrior-ws webwarrior-ws force-pushed the wip/context-lens-continued branch from d6731dc to a868f42 Compare June 16, 2026 10:59
webwarrior-ws and others added 14 commits June 16, 2026 13:01
Added a --context-lens (-c) flag, that installs context-lens
after installing Pi, and creates a launcher script "cpi" for it.

context-lens is installed from git repository, with 2 patches
applied:
- Enable using PPQ.ai in Pi through MITM proxy.
- When launching pi, add providers based on env. vars &
  auth.json.

Launcher script launches context-lens with `pi` argument.
Context-lens sets up pi (creates temp pi directory with
modified settings), in particular to route its traffic through
mitmproxy. Otherwise context-lens wouldn't be able to capture
requests. Launcher script can't use `spi` because `pi` argument
is not an executable name but a switch that tells context-lens
to perform mentioned pi setup.
Otherwise there would be a permission error on fresh install:
```
fatal: could not create work tree dir 'context-lens': Permission denied
Error: Error: command 'git' failed (exit code 128)
```
To have less changes in the PR, because now cpi script is
calling spi and thus createContextLensLauncherScript no longer
needs to shares code with createLauncherScript.

There are still some changes to createLauncherScript due to
forwarding of proxy env. vars.
And if true, then just launch pi using spi wrapper.

But that other pi instance's requests won't be captured by
context-lens.
Now extra pi instnaces launched using cpi script route their
requests through contex-lens mitm proxy so the requests are
captured. However they are all lumped together as if it was a
single session.
@webwarrior-ws webwarrior-ws force-pushed the wip/context-lens-continued branch from a868f42 to b36b17f Compare June 16, 2026 11:03
@webwarrior-ws webwarrior-ws force-pushed the wip/context-lens-continued branch from 43c4cd5 to 1206ab4 Compare June 17, 2026 08:46
@knocte

knocte commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author
  • Remove MitmProxy from README.
  • Add a TMP commit that makes --extensions flag point to your awto-pi-lot's fork, so that I can test this end2end.

@knocte

knocte commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Also let's fix CI.

Instead use context-lens with regular proxy and set
UPSTREAM_OPENAI_URL to https://api.ppq.ai. This will only
work after [1] is merged.

[1] tarsgate/awto-pi-lot#44
@webwarrior-ws webwarrior-ws force-pushed the wip/context-lens-continued branch from 49cde1f to d807d60 Compare June 18, 2026 10:38
@webwarrior-ws

Copy link
Copy Markdown
Collaborator

We also have to abandon the idea of reusing spi wrapper in cpi because context-lens has to receive pi argument to prepare pi directory with modified models.json.

@knocte

knocte commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

ok

That includes tarsgate/awto-pi-lot#44
in order to be able to test this PR.
Because context-lens has to receive "pi" argument to prepare pi
directory with modified models.json.
@webwarrior-ws webwarrior-ws force-pushed the wip/context-lens-continued branch from 3ada093 to c483e2e Compare June 18, 2026 12:20
Comment thread src/index.ts
contextLensDir: string,
apiKeyExport: Option<{ name: string; value: string }> = Nothing
): Promise<void> {
const cmd = `HOME=${agentUserHome} UPSTREAM_OPENAI_URL=https://api.ppq.ai node ${contextLensDir}/dist/cli.js pi`;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@webwarrior-ws what is this? skynot is not PPQ related

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I raised this in Telegram group, and you said "I don't care about that for now".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants