Skip to content

feat(probe): discover probe slots in adapter libraries - #170

Open
IRENEKO wants to merge 2 commits into
generative-computing:mainfrom
IRENEKO:feature/probe-library-discovery
Open

IRENEKO wants to merge 2 commits into
generative-computing:mainfrom
IRENEKO:feature/probe-library-discovery

Conversation

@IRENEKO

@IRENEKO IRENEKO commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Probe slots could only be composed from a YAML manifest. A library can now ship a
probe the same way it ships a LoRA, so --adapters <library> picks it up:

<name>/<target_model>/probe/
├── probe_head.safetensors
├── probe_config.json   # {"labels": ["no", "yes"], "layer": 24}
└── io.yaml

The slot is named <name>_probe, so a probe can sit next to a LoRA of the same
name (in one library, or across two libraries composed together). layer is
optional; omitted means the final post-norm state, as in the manifest.

Changes:

  • discover_adapters accepts probe as a technology and requires the probe files
    instead of adapter_model.safetensors / adapter_config.json. Probes are left out
    of the module-contribution report, which reads adapter_config.json.
  • Compose reads labels and layer from each discovered probe's probe_config.json
    into the same probe_meta the manifest path fills.
  • The compose report skips LoRA analysis when every slot is a probe. A probe-only
    compose used to crash there (KeyError: 'adapter_names').

Hub libraries need no download change: a probe/ folder falls under the existing
<name>/<model>/** fallback pattern.

Testing

  • tests/composer/test_adapter_filtering.py: a LoRA and a probe under one name are
    both discovered, as ans and ans_probe.
  • tests/composer/test_selective_download.py: a Hub probe folder is in the download
    patterns.
  • tests/composer/test_probe_compose.py: two CLI end-to-end composes. One is a
    probe-only library (granite-4.0-micro). The other is answerability/.../lora/ plus
    answerability/.../probe/ (granite-4.1-3b), which gives slots answerability and
    answerability_probe.
  • Passed on an H100: composer CPU tests 42/42, test_probe_compose.py 14/14,
    test_compose_e2e.py 9/9.

A library can now ship a probe as <name>/<model>/probe/ with
probe_head.safetensors and probe_config.json (labels, layer), so
--adapters <library> composes it without a manifest. The slot is named
<name>_probe, so it can sit next to a LoRA of the same name. The compose
report skips LoRA analysis when every slot is a probe, which used to crash.

Signed-off-by: IRENEKO <cyko@ibm.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@AlonMalach

AlonMalach commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Thanks Irene! I went over the PR, and it looks good

One small gap, in case you'd like to cover it: from a Hub library, when one adapter name has both a LoRA folder (lora/ or alora/) and a probe/ folder, only the LoRA folder is downloaded, so <name>_probe silently ends up missing from the composed model.

Why only from the Hub

  • Local library: compose scans the folder directly, and every file is already there, so discovery sees both lora/ and probe/.
  • Hub library: compose downloads first, then scans what it downloaded. To save bandwidth, the download fetches only selected folders, and probe/ is not selected when the same name has a LoRA.

Where in the code

  1. adapter_discovery.py:365: _resolve_technology asks the Hub which folders exist under <name>/<model>/, keeps only aLoRA/LoRA, and returns one of them (aLoRA preferred):
    subtree = list_repo_tree(
        repo_id,
        repo_type="model",
        path_in_repo=f"{adapter_name}/{target_model_name}",
    )
    technologies = {
        item.path.split("/")[-1]
        for item in subtree
        if isinstance(item, RepoFolder)
        and item.path.split("/")[-1] in ("alora", "lora")
    }
  2. adapter_discovery.py:425: _build_allow_patterns then writes one download rule per adapter name, for that single technology:
    tech = _resolve_technology(repo_id, name, target_model_name)
    if tech:
        patterns.append(f"{name}/{target_model_name}/{tech}/**")
    else:
        # Model not found for this adapter — include anyway so
        # discover_adapters can report it as missing downstream
        patterns.append(f"{name}/{target_model_name}/**")

So for answerability/granite-4.1-3b/ holding lora/ and probe/, the only rule is answerability/granite-4.1-3b/lora/**. Only the lora/ files arrive, discovery never sees probe/, and no answerability_probe slot is built, without any warning. A probe under its own name works: no aLoRA/LoRA is found, so the rule falls back to the whole <name>/<model>/** folder. That's the case test_probe_dir_is_downloaded covers.

Since probes and adapters should normally have different names and be trained on different data, I don't see this as blocking. If you want to close it: at line 365, keep probe in the listing, and at line 425, add a second rule, <name>/<model>/probe/**, when a probe/ folder exists. A download test with lora and probe under one name would cover it. Up to you: here, in a follow-up, or not at all.

From a Hub library only the preferred LoRA folder of a name was
downloaded, so a probe/ under the same name was silently dropped.
_resolve_technology now also returns probe, and each technology gets
its own download pattern.

Signed-off-by: IRENEKO <cyko@ibm.com>
@IRENEKO

IRENEKO commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @AlonMalach ! I pushed a new commit and revised at the two places you pointed to.

  • _resolve_technology now also lists probe, and returns the preferred LoRA variant (aLoRA over LoRA) plus probe when present.
  • _build_allow_patterns writes one download pattern per technology, so a name with lora/ and probe/ gets both <name>/<model>/lora/** and <name>/<model>/probe/**.

I added test_probe_next_to_lora_of_same_name_is_downloaded, which covers a name with alora/, lora/ and probe/. Also checked it against a Hub with a private test library: the RAG citations LoRA in citations/granite-4.1-3b/lora/, plus a dummy probe head in citations/granite-4.1-3b/probe/. Before the fix, compose built only citations. After it, compose downloaded both folders and built citations and citations_probe.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants