Skip to content

fix(update): walk the include graph so a transitive @branch cache can't hide behind a green check (0yc) - #317

Merged
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
lane/cli-0yc
Sep 6, 2026
Merged

Brian Krabach (bkrabach) merged 1 commit into
mainfrom
lane/cli-0yc

Conversation

@bkrabach

Copy link
Copy Markdown
Collaborator

Defect

amplifier update and amplifier bundle update both printed a green "All sources up to date" while a cache in the active closure sat behind its @main upstream.

Measured on this host 2026-09-06: ~/.amplifier/cache/amplifier-d1dda27a16518560 (git_url https://github.com/microsoft/amplifier, ref main) was pinned at c03a88b while upstream main was 28588b9 — 28588b9 was not even a fetched object there. microsoft/amplifier is not a registered source; it enters the closure only through amplifier-foundation's bundle.md:16:

- bundle: git+https://github.com/microsoft/amplifier@main#subdirectory=behaviors/amplifier-expert.yaml

Root cause (found first, as required)

Two independent facts combine into a permanent silent staleness.

1. Enumeration is registration-shaped, not closure-shaped.

amplifier_app_cli/commands/update.py:364 _check_all_bundle_status() enumerates exactly two things: discovery.list_cached_root_bundles() and AppSettings().get_app_bundles(). list_cached_root_bundles() keeps only registry entries with is_root: true — amplifier_app_cli/lib/bundle_loader/discovery.py:679-685 sorts every non-root entry into nested_bundles, and discovery.py:496-497 discards that half. A #subdirectory= include registers with is_root: false (amplifier_foundation/registry.py:520-545), so a repo whose only registry presence is such an entry is checked by nobody. On the day of the report, amplifier-expert-behavior was registered (is_root: false, root_name: "amplifier") with no amplifier root entry — that entry only appeared later, at 08:43, after the cache had already been refreshed by other means at 07:48.

amplifier bundle update fails the same way from the other end: check_bundle_status() → _collect_source_uris() (amplifier_foundation/updates/__init__.py:138-140) deliberately skips includes on the comment "Included bundles are now registered as first-class bundles and will be checked independently." That premise does not hold for an include with no root entry.

2. The loader never refreshes a cached @<branch> include — no TTL, ever.

amplifier_foundation/sources/git.py:697-709: GitSourceHandler.resolve() returns the existing cache verbatim whenever it passes _verify_clone_integrity(). There is no fetch, no age check, no revalidation. Refresh happens only via GitSourceHandler.update() (git.py:877-899), which only the update commands call.

So the answer to "does the loader ever refresh on session start?" is never — which is what makes a missed enumeration permanent rather than merely late. That decides the fix location: the update side, here. The foundation side is filed separately as recipes-72y (naming updates/__init__.py:138-140 and the exact fix), because fixing check_bundle_status() upstream would make every host correct, not just this CLI.

Change

New amplifier_app_cli/utils/include_graph.py — a cycle-safe BFS over includes::

  • Reuses the loader's resolution, does not reimplement it: BundleRegistry._parse_include, _resolve_include_source (handles @namespace:path, git+, #subdirectory= identically to a real session), _load_from_path to parse bundle files, GitSourceHandler._get_cache_path / get_status / update, and foundation's own SourceStatus.is_pinned.
  • Read-only walk: reads caches that already exist and never resolves a missing one — resolving downloads, and a download would make a deleted cache report as healthy, the exact side effect _check_all_bundle_status was written to avoid.
  • Cycle-safe on the full URI (fragment included, so two behaviors in one repo are not collapsed).
  • Skips anything already covered by a direct row — a registered repo is not "transitive", and two rows for one cache would misreport which carries the update.

Wired into both commands:

before after
update.py:_check_all_bundle_status registry roots + app bundles + _check_transitive_bundle_status()
update.py Bundles table transitive rows absent indented ↳ <name> (via <parent>), grouped under the registered root, cached vs remote SHA
update.py executor n/a transitive rows refreshed via GitSourceHandler.update()
bundle.py:_bundle_update_async / _bundle_update_all_async check_bundle_status() only + _augment_with_transitive() / _refresh_transitive()

Pinned @sha / @v1.2.3 includes render pinned in the Remote column and are never refreshed (get_status() already reports has_update=False for them). Because transitive rows fold into bundle_results, has_bundle_updates picks them up automatically — "All sources up to date" can no longer print over a stale transitive cache.

Tests

tests/test_transitive_include_updates.py — 12 tests, real local git repos over git+file:// so cache-key derivation, git ls-remote comparison and re-clone all run for real:

  1. B registered / C not → C checked, attributed to B, correct stale-vs-remote SHAs
  2. refresh actually advances C's cache
  3. a directly-registered repo is never duplicated as a transitive row
  4. pinned @sha reported pinned, has_update=False, cache unmoved
  5. pinned @v1.0.0 tag likewise
  6. cycle (B ↔ C) terminates, C found once
    7–8. table renders ↳ … (via parent) with both SHAs; pinned row shows pinned
  7. executor refreshes a transitive row via the git handler
  8. --check-only does not print "All sources up to date" over a stale transitive row
  9. _check_all_bundle_status() end-to-end — verified to FAIL with the walk disabled: AssertionError: the include walk is not wired into the enumeration … Got: ['b']

Gates

  • uv run pytest -q → 1877 passed, 1 skipped, 13 deselected, 1 xfailed
  • uv run ruff check on all four touched files → All checks passed
  • Windows: the git+file:// fixtures are skipif'd (drive-letter URIs take a different resolution path); the behaviour under test is platform-independent and covered on POSIX runners. The render/executor tests run everywhere.

Live proof (this host, isolated HOME + AMPLIFIER_HOME copy)

Isolated because refreshing against the real ~/.amplifier would destroy the reproduction. The copy carries the real registry.json, settings.yaml and full 403 MB cache; ~/.amplifier/cache/amplifier-d1dda27a16518560 was restored to c03a88b with matching .amplifier_cache_meta.json, and the (later-added) amplifier root entry removed — i.e. exactly the registry state at the time of the report.

Before — installed CLI (amplifier update --check-only):

│ foundation            │ 2ef5e12 │ 2ef5e12 │ ✓ │
...
Legend: ✓ up to date  ● update available  ◦ local changes
✓ All sources up to date

microsoft/amplifier appears nowhere in the table. Cache: c03a88b.

After — this branch (amplifier update --check-only):

│ foundation                     │ 2ef5e12 │ 2ef5e12 │ ✓ │
│   ↳ amplifier (via foundation) │ c03a88b │ 28588b9 │ ● │
...
Updates available:
  • 1 bundle(s)

Apply — yes | amplifier update:

BEFORE: c03a88b Merge pull request #396 from microsoft/lane/am-content
✓ Update complete
  ✓ Bundle: git+https://github.com/microsoft/amplifier@main
AFTER:  28588b9 Merge pull request #397 from microsoft/lane/am-7vl
{"cached_at": "2026-09-06T10:32:40.613082", "ref": "main",
 "commit": "28588b93886dd4b134f131294ca98e944d71cbfd",
 "git_url": "https://github.com/microsoft/amplifier"}

The other command — amplifier bundle update foundation:

Installed CLI, cache at c03a88b: All sources are up to date.

This branch:

│ git+https://github.com/microsoft/amplifier@main │ 🔄 Update │ via foundation c03a88b → 28588b9 │
...
✓ Updated (transitive): git+https://github.com/microsoft/amplifier@main
AFTER:  28588b9 Merge pull request #397 from microsoft/lane/am-7vl

The measured symptom, fixed, on both commands.

Filed alongside

  • recipes-72y — foundation: check_bundle_status() drops includes on a false premise (updates/__init__.py:138-140), with the exact fix.
  • recipes-j48 — app-cli: bundle discovery hardcodes Path.home()/.amplifier/registry.json (discovery.py:668, :537, paths.py:47), ignoring AMPLIFIER_HOME. Latent split-brain, surfaced while building the isolated-home proof.

Work item: recipes-0yc

Generated with Amplifier

Co-Authored-By: Amplifier 240397093+microsoft-amplifier@users.noreply.github.com

…'t hide behind a green check (0yc)

Both `amplifier update` and `amplifier bundle update` enumerated update
targets from REGISTERED sources only. A bundle reaching the active closure
solely through another bundle's `includes:` was in neither the registry-root
list nor app-bundle settings, so its cache was never checked -- and both
commands printed "All sources up to date" over it.

Foundation's loader never refreshes a cached `@<branch>` source on its own
(sources/git.py:697-709: a cache hit returns verbatim, no TTL, no fetch), so
a source the update path does not enumerate is a source that never updates.

Adds a cycle-safe include walk that reuses the loader's own resolution
(`_parse_include` / `_resolve_include_source` / `_load_from_path`), reads
only caches that already exist -- resolving would download and make a
deleted cache look healthy -- and refreshes each transitive `git+...@branch`
with the same `GitSourceHandler.update()` a direct refresh bottoms out in.
Pinned `@sha` / `@tag` includes are reported as pinned, never moved.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach
Brian Krabach (bkrabach) merged commit c120a36 into main Sep 6, 2026
9 checks passed
@bkrabach
Brian Krabach (bkrabach) deleted the lane/cli-0yc branch September 6, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants