Problem
The modules under models/ look organized by API protocol, but the layout
mixes axes: openai.py / anthropic.py / gemini.py are protocol-shaped,
alcf_endpoints.py is gateway-shaped, groq.py is vendor-shaped, codex.py
is product-shaped.
Two consequences:
ChatOpenAI is built in five places, two outside models/:
openai.py:300/:317, alcf_endpoints.py:120, agent/turn.py:294,
agent/llm_agent.py:350.
- Dispatch exists twice — the same
if/elif chain in
models/loader.py:77-113 and agent/llm_agent.py:298-336.
The root cause is that two independent dimensions — which protocol to speak,
and which endpoint to speak to — are collapsed into one model-name prefix.
Proposal
models/
protocols/ openai_compatible.py anthropic_native.py google_native.py
endpoints/ argo.py alcf.py openrouter.py vllm.py
openai_direct.py anthropic_direct.py google_direct.py
groq.py local_model.py codex.py # own clients, unchanged
loader.py # the only dispatch site
Endpoint specs declare only the differences — base URL, auth style,
model-name transform, per-model quirks, default params — and reuse the protocol
layer to build the client. One endpoint can then hold several specs (Argo two, ALCF three) and one protocol can back several endpoints.
Worth evaluating whether LLM Rosetta can provide the protocol layer here rather
than us maintaining our own wrappers.
Problem
The modules under
models/look organized by API protocol, but the layoutmixes axes:
openai.py/anthropic.py/gemini.pyare protocol-shaped,alcf_endpoints.pyis gateway-shaped,groq.pyis vendor-shaped,codex.pyis product-shaped.
Two consequences:
ChatOpenAIis built in five places, two outsidemodels/:openai.py:300/:317,alcf_endpoints.py:120,agent/turn.py:294,agent/llm_agent.py:350.if/elifchain inmodels/loader.py:77-113andagent/llm_agent.py:298-336.The root cause is that two independent dimensions — which protocol to speak,
and which endpoint to speak to — are collapsed into one model-name prefix.
Proposal
Endpoint specs declare only the differences — base URL, auth style,
model-name transform, per-model quirks, default params — and reuse the protocol
layer to build the client. One endpoint can then hold several specs (Argo two, ALCF three) and one protocol can back several endpoints.
Worth evaluating whether LLM Rosetta can provide the protocol layer here rather
than us maintaining our own wrappers.