Repository navigation
Commit f166328
routing show: a candidate is active only if its glob resolves against live model lists (#323)
* fix(routing show): a candidate is active only if its glob resolves live
"Configured" is a property of the PROVIDER; a candidate's model is a GLOB
that must ALSO match something that provider actually lists. `show` marked a
candidate active on provider presence alone, so it claimed models the session
could not serve. Measured 2026-09-07 in a DTU with only gpt-5.6-terra
configured on openai:
fast -> * openai / gpt-?.?-luna <- active # nothing would resolve this
Fix: `_LiveModels` -- one memoized list_models() per DISTINCT matrix provider
name, fetched lazily (only for candidates actually evaluated) and shared
across roles, so a typical matrix costs a handful of calls. Alias-aware: a
`provider: openai` candidate lists models from whichever openai-family
backend is configured, canonical first -- the resolver's own preference. The
resolved id is shown (`<- active (gpt-5.6-luna)`; `gpt-?.?-terra ->
gpt-5.6-terra` in the table) so users see what will actually run. The id is
chosen with the hook's own version-aware sort when importable, so display and
runtime agree.
Honest about what it cannot check. A provider whose list could not be fetched
-- offline, no key, module not installed, empty list -- is UNVERIFIED, never
"no match": it renders exactly as before this change and is named once in a
footer. A glob that matches nothing a LISTED provider returns is "no model
matches this glob" and the display falls through to the next candidate, as
the resolver would. `_resolve_role` grows an optional `live` parameter; the
`routing list` compatibility counts do not pass it and stay offline-fast and
byte-identical.
Two things found while making it real on this host:
1. WALL-CLOCK CAP. A provider with no usable credentials does not fail fast:
provider-anthropic retries list_models() 5 times with backoff, measured at
~35s. `_list_models_bounded` runs each listing on a worker thread with a
6s cap (AMPLIFIER_ROUTING_LIST_TIMEOUT overrides); a miss is UNVERIFIED.
Without this the routing test files went from 3s to 578s, and a user with
one dead key would have waited half a minute per role.
2. `_list_models_for_provider` RETURNED REPRS, NOT IDS. ModelInfo carries
`.id`; the helper read `getattr(m, "name", m)`, fell through to str(m),
and returned `"id='gpt-5.6-terra' display_name=..."` -- a list that
matches no glob. It had no caller until now, so it went unnoticed; the
first live run showed "no model matches" for every openai, anthropic and
gemini candidate on a host where all three resolve. Fixed to read `.id`.
tests/conftest.py gains an autouse guard that stubs `_list_models_bounded`
to "could not list" -- unit tests must never do live listing, and the stub
makes every glob unverified, i.e. the pre-change output. The verification
tests patch the same seam with an explicit table on top of it.
Verified live on this host (15 provider instances): every openai glob
resolves to its terra/luna id, gemini resolves, the whole detailed view takes
2.7s.
Tests: tests/test_routing_show_verifies_globs.py (15) -- live match is
active with the id; nothing-listed is no-match; exact names skip the lookup;
unlistable is unverified and named; unconfigured never lists; picks the same
id the hook would (suffix-free glob does not fall for -fast); one call per
provider across roles; alias lists from the configured backend; _resolve_role
falls through a no-match like the resolver, is unchanged without `live`, and
yields None when nothing matches; details view marks no-match and promotes
the next candidate; unverified renders as before plus footer; table shows the
resolved id; the helper returns ids not reprs. Full suite 1932 passed in 19s,
ruff clean on touched files.
Generated with Amplifier
Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
* test(conftest): private MonkeyPatch in the autouse guard -- fixes Windows teardown
The autouse `no_live_model_listing` guard requested the shared `monkeypatch`
fixture. An autouse fixture is set up before every test's own fixtures, so
that made `monkeypatch` set up BEFORE -- and torn down AFTER --
`temp_context_dirs` in tests/lib/mention_loading/test_resolver.py. Two tests
there `monkeypatch.chdir()` into a TemporaryDirectory: teardown now deleted
the directory while it was still the process cwd. Linux/macOS allow that;
Windows raises PermissionError ("being used by another process"),
deterministically, on both py3.11 and py3.12 (CI for #323, twice).
Fix: a private `pytest.MonkeyPatch.context()` whose lifetime is this
fixture's own. --setup-plan for test_resolver_cwd_file now shows the original
order: TEARDOWN monkeypatch, then TEARDOWN temp_context_dirs.
Generated with Amplifier
Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
---------
Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>1 parent ea87c0c commit f166328
3 files changed
Lines changed: 568 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
447 | 448 | | |
448 | 449 | | |
449 | 450 | | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
450 | 615 | | |
451 | | - | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
452 | 619 | | |
453 | 620 | | |
454 | 621 | | |
455 | 622 | | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
456 | 628 | | |
457 | 629 | | |
458 | 630 | | |
459 | 631 | | |
460 | | - | |
461 | | - | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
462 | 638 | | |
463 | 639 | | |
464 | 640 | | |
| |||
724 | 900 | | |
725 | 901 | | |
726 | 902 | | |
727 | | - | |
728 | | - | |
729 | | - | |
730 | | - | |
731 | | - | |
732 | | - | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
733 | 936 | | |
734 | 937 | | |
| 938 | + | |
735 | 939 | | |
736 | 940 | | |
737 | 941 | | |
| |||
776 | 980 | | |
777 | 981 | | |
778 | 982 | | |
| 983 | + | |
779 | 984 | | |
780 | 985 | | |
781 | 986 | | |
| |||
786 | 991 | | |
787 | 992 | | |
788 | 993 | | |
| 994 | + | |
789 | 995 | | |
790 | 996 | | |
791 | 997 | | |
| |||
800 | 1006 | | |
801 | 1007 | | |
802 | 1008 | | |
| 1009 | + | |
| 1010 | + | |
| 1011 | + | |
| 1012 | + | |
| 1013 | + | |
| 1014 | + | |
803 | 1015 | | |
804 | | - | |
| 1016 | + | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
| 1020 | + | |
| 1021 | + | |
| 1022 | + | |
| 1023 | + | |
805 | 1024 | | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
806 | 1030 | | |
807 | 1031 | | |
808 | | - | |
| 1032 | + | |
809 | 1033 | | |
810 | 1034 | | |
811 | 1035 | | |
| |||
818 | 1042 | | |
819 | 1043 | | |
820 | 1044 | | |
821 | | - | |
822 | | - | |
823 | | - | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
| 1050 | + | |
| 1051 | + | |
| 1052 | + | |
| 1053 | + | |
| 1054 | + | |
| 1055 | + | |
824 | 1056 | | |
825 | 1057 | | |
826 | 1058 | | |
| |||
1246 | 1478 | | |
1247 | 1479 | | |
1248 | 1480 | | |
1249 | | - | |
| 1481 | + | |
| 1482 | + | |
| 1483 | + | |
| 1484 | + | |
| 1485 | + | |
| 1486 | + | |
| 1487 | + | |
| 1488 | + | |
| 1489 | + | |
1250 | 1490 | | |
1251 | 1491 | | |
1252 | 1492 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
35 | 66 | | |
36 | 67 | | |
37 | 68 | | |
| |||
0 commit comments