Skip to content

docs(skills): tell agents how malloy-config.json must be set for local data - #1052

Closed
jswir wants to merge 1 commit into
malloydata:mainfrom
jswir:skills/local-data-config
Closed

docs(skills): tell agents how malloy-config.json must be set for local data#1052
jswir wants to merge 1 commit into
malloydata:mainfrom
jswir:skills/local-data-config

Conversation

@jswir

@jswir jswir commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

The problem

A package Publisher serves correctly can fail to compile in the VS Code extension or the Malloy CLI, on the same machine, from the same files. Nothing in the skills said why, so an agent hitting it reads the cascade of 'org_slug' is not defined errors and starts editing the model.

Two things combine:

  1. Different resolution base. Publisher resolves a relative duckdb.table('data/x.csv') against the package root on its own. The CLI and the extension resolve it against the DuckDB connection's workingDirectory, read from malloy-config.json - a file no skill mentioned.
  2. A relative workingDirectory resolves against the process cwd. canonicalizeConfigPath (malloy-db-duckdb/src/duckdb_config.ts) calls canonicalizePath with no baseDirectory, so it is a bare path.resolve(input). Not the config file's directory, and not the VS Code workspace root.

So the same config compiles from one directory and fails from another. That is what makes this feel intermittent, and what makes it look like a model bug rather than a config one.

Verified, not inferred

malloy-cli reads the same malloy-config.json the extension does, which makes it a real oracle. Against a package with the model in pkg/ and data in pkg/data/:

workingDirectory cwd result
"malloy" (relative) repo root compiles, returns rows
"malloy" (relative) /tmp IO Error: No files found that match the pattern "data/product_usage.csv", then one "not defined" error per field
absolute /tmp compiles, returns rows

The middle row is the reported symptom, reproduced.

The change

Three files, one concept, no definition duplicated - the mechanism lives in one place and the other two point at it.

  • malloy-getting-started gains a setup step that has the agent ask whether the CLI or the VS Code extension is also in use, then gives the config to write. Setup-time prevention, which is the only point where this is cheap.
  • malloy-gotchas-modeling carries the mechanism, beside the existing read-a-file-in-place sections, because that is the file an agent reads before writing duckdb.table(...).
  • malloy-debug gets one quick-fix row mapping the error to the fix, and states that the field errors beneath it are cascade - so nobody starts renaming columns.

It also records what is not this case: a remote Credible connection arrives as a publisher proxy connection that the extension supplies, where no workingDirectory applies at all. Since that came up in the same breath, the note also mentions the documented token caveat - user-scoped, short-lived, unrefreshed - so auth failures late in a long session are not misread as a broken model.

Notes for the reviewer

  • Docs only. No code, no frontmatter, no reference/ files, so test:skills (frontmatter stamping, name-matches-directory, reference shipping) is untouched.
  • Dash style matches the three files, which use - and no em-dashes.
  • The recommendation is "make it absolute" rather than teaching a relative form, because there is no relative form that survives a cwd change. If there is a host-injected rootDirectory default meant to cover this (src/common/connections/CONTEXT.md mentions workingDirectory: {default: {config: 'rootDirectory'}}), then the better fix is in the extension and this doc should say so instead - happy to change tack.

…l data

A package that Publisher serves correctly can fail to compile in the VS Code
extension or the CLI, on the same machine, from the same files. Publisher
resolves a relative `duckdb.table('data/x.csv')` against the package root
itself; the other two hosts resolve it against the DuckDB connection's
`workingDirectory` from `malloy-config.json`, which nothing was telling anyone
to set.

Worse, a *relative* `workingDirectory` is resolved against the process's cwd:
`canonicalizeConfigPath` (malloy-db-duckdb/src/duckdb_config.ts) calls
`canonicalizePath` with no `baseDirectory`, so it is a bare
`path.resolve(input)` - not the config file's directory, and not the VS Code
workspace root. The same config therefore compiles from one directory and fails
from another, which is what makes this look intermittent and look like a model
bug.

Verified with malloy-cli, which reads the same config file, against a package
whose model lives in `pkg/` and data in `pkg/data/`:

  relative "malloy", cwd = repo root  -> compiles, returns rows
  relative "malloy", cwd = /tmp       -> IO Error: No files found that match
                                         the pattern "data/product_usage.csv",
                                         then a "not defined" error per field
  absolute path,     cwd = /tmp       -> compiles, returns rows

Three edits, one concept, no duplicated definition:

- malloy-getting-started gains a setup step that has the agent ASK whether the
  CLI or the extension is also in use, then gives the config. It also records
  that a remote Credible connection is not this case: that arrives as a
  `publisher` proxy connection the extension supplies, where no
  `workingDirectory` applies, and whose tokens are short-lived and unrefreshed.
- malloy-gotchas-modeling carries the mechanism, next to the other
  read-a-file-in-place sections, since that is what an agent reads before
  writing `duckdb.table(...)`.
- malloy-debug gets one row mapping the error to the fix, and says the field
  errors under it are cascade - so nobody starts renaming columns.
@jswir jswir closed this Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant