detect() writes converted sidecars into the scanned tree; cache_root does not redirect them

[sortakool]

Summary

detect() writes into the tree it is scanning. When the root contains an Office
or Google-Workspace document, detection converts it and writes the markdown
sidecar to <root>/graphify-out/converted/. That makes detect() a mutating
operation on its own input, which breaks any caller that treats the scanned tree
as immutable.

Observed on 0.9.44.

Why this is a problem, not a quirk

We scan pinned third-party sources in disposable clones and assert afterwards
that the clone is byte-identical to the commit we materialized — the check exists
so that a detector can never silently alter reviewed source. Detection itself
trips that assertion:

disposable clone changed during detection: 2 dirty path(s):
  ['?? graphify-out/converted/example_8a530a1d.md',
   '?? graphify-out/converted/example_d1c39225.md']

The two sidecars come from cognee/tests/test_data/example.docx and
example.pptx. It is fully deterministic: every source shipping an Office
document drifts, and no other source does.

More generally, a read-shaped API that writes to its argument is surprising for
read-only mounts, immutable checkouts, concurrent scans of one tree, and anything
doing provenance or reproducibility checks.

There is no way to redirect it

converted_dir is hardcoded (detect.py, 0.9.44):

converted_dir = root / GRAPHIFY_OUT / "converted"

detect() does accept cache_root, which reads like the knob for this, but it
only reaches _cache.cached_word_count(...) — the word-count cache. Nothing
routes conversion output through it.

Reproduce

mkdir -p repro && cd repro && git init -q
cp /path/to/any.docx .
git add -A && git commit -qm fixture
python -c "from pathlib import Path; from graphify.detect import detect; detect(Path('.'))"
git status --porcelain --untracked-files=all
# ?? graphify-out/converted/any_<hash>.md

Suggested resolution

Any of these would resolve it for us, in preference order:

1. Honour cache_root for conversion output as well, so a caller can direct
   sidecars outside the scanned tree.
2. Add an explicit converted_dir / out_dir parameter to detect().
3. Make conversion opt-in during detect(), given that detection's documented
   job is classification rather than materialization.

Happy to send a PR if you have a preference on the shape.

[sortakool]

Flagging that #2794 ("fix(detect): respect cache root for converted sidecars") looks like it addresses this — it declares Fixes #2787 and redirects converted sidecars to <cache_root>/graphify-out/converted/, which matches the behaviour described here.

It is still open / unmerged at the time of writing, so this is a pointer rather than a resolution.

One data point that may be useful: the fix is not in the released 0.9.45. Checked against the installed wheel:

• detect() does accept cache_root — graphify/detect.py:1460 — and threads it into the word-count cache at :1500.
• But the sidecar destination is still anchored to the scanned root: converted_dir = root / GRAPHIFY_OUT / "converted" at graphify/detect.py:1647.

So on 0.9.45 detect() can still write into a source tree that was passed a separate cache_root.

Context for why we noticed: we pin graphify sources at an exact commit and verify the tree digest before extracting, so a detect() run that writes graphify-out/ into the scanned checkout shows up as source drift against a pinned SHA. We currently work around it by materialising each source into a disposable temporary tree before scanning.

Happy to re-check against a release once #2794 lands, if that is useful.

[safishamsi]

Fixed in v0.9.47: https://github.com/Graphify-Labs/graphify/releases/tag/v0.9.47 (converted sidecars now write to the cache root, not the source tree). Closing as resolved.