fix(orchestrator): gate unauthenticated plugin tools from the orchestrator's tool surface (#474)

[Weegy]

Summary

Closes the gap described in #474: an unauthenticated / not-yet-connected plugin's tools were previously offered to (and dispatchable by) the orchestrator, causing a wasted round-trip that fails with OAuthTokenError/status errors on the first real call. The orchestrator now gates every path that offers or dispatches a plugin's tools on that plugin's actual readiness, derived from two independent signals:

• PluginStatusRegistry — explicit, set by a plugin via ctx.status.report({state:'needs_action'|'error'}).
• OAuthReadinessTracker (new) — automatic, derived from the same vault token state ctx.oauthTokens reads, for type:'oauth' plugins that never call ctx.status.report() themselves. Recomputes freshness/refreshability against the wall clock on every read, not just at plugin-activation time.

Both are ANDed together and consulted at every gate site closed over the review rounds below:

• Tool-list assembly (Orchestrator.buildToolsList), including the hardcoded memory tool fast path
• Tool dispatch (Orchestrator.dispatchToolInner), re-checked at invocation time to close the list-vs-dispatch race window
• The standalone ToolDispatchService (subscription-CLI provider / loopback-MCP bridge), a separate entry point gated independently
• System prompt promptDoc splicing and the "Fach-Agenten" domain-tool roster
• DirectLine #token resolution (degrades to the existing "Specialist … is no longer available." notice instead of leaking an internal error string)
• Forced tool_choice (directLineObligationState, the #332 primitive)
• ctx.tools.registerHandler() (e.g. the Anthropic-native memory tool), which previously bypassed the agentId-based gate entirely by using a different registration API than ctx.tools.register()
• OAuth-token expiry (a stored token that exists but is expired with no refresh token is not "ready")

Deliberately separate from the MCP-server-specific auth-gap flow (mcpOAuthService), which already handles unauthenticated MCP servers on its own.

Review history

This branch went through 12 rounds of adversarial + cross-vendor (codex) review before reaching this state. Each round closed one real, concrete gap a prior round's audit had missed — the commit history below is a genuine iterative-hardening trail, not padding:

1. Native tools gated at list + dispatch time (buildToolsList, dispatchToolInner), plus the standalone ToolDispatchService.
2. getSystemPrompt()'s promptDoc collection wasn't gated — a hidden tool's documentation still reached the model.
3. Domain tools (DomainTool, e.g. query_<slug>) weren't gated at all — a second tool-registration path the initial fix missed.
4. The "Fach-Agenten" roster and PluginStatusRegistry.isReady()'s correctness (previously dependent on caller discipline, not its own invariant) plus forced tool_choice and DirectLine #token resolution.
5. The entire mechanism only engaged when a plugin explicitly called ctx.status.report() — the repo's own generic OAuth Connect flow never does this automatically, so a standard type:'oauth' plugin stayed exempt until Connect completed. Added OAuthReadinessTracker, deriving readiness from the same vault state ctx.oauthTokens already reads.
6. Branch fell behind origin/main, which had since gained a commit touching the same file — merged and re-verified.
7. registerHandler() (a second, public plugin-api tool-registration entry point, used by harness-memory/harness-memory-postgres) didn't carry an agentId, so a plugin using it instead of register() bypassed the gate.
8. OAuthReadinessTracker checked token presence, not usability — an expired, non-refreshable token still read as "connected". Extracted the exact expiry/refreshability logic ctx.oauthTokens.get() already used into a shared helper so the two can't drift.
9. The hardcoded memory tool fast path in buildToolsList/dispatchToolInner bypassed the registry gate entirely, regardless of round 7's fix.
10. docs/middleware-agent-handoff.md needed a section describing the new cross-cutting mechanism, per this repo's own documentation policy.
11. OAuthReadinessTracker.isConnected() cached its verdict at plugin-activation time and never re-evaluated it against the wall clock afterward — a token that went stale mid-session (crossed into its refresh margin) kept reading as ready until the next activation. Now recomputes freshness fresh on every read, mirroring how ctx.oauthTokens.get() itself never caches a verdict.
12. Final merge of origin/main (one more commit landed, unrelated, no code overlap — only a docs/CHANGELOG.md [Unreleased]-section conflict, resolved by keeping both entries) and full re-verification.

A recurring false-positive flag (files from attachmentExtract.ts/codegen.ts/healthScore.ts/agentSpec.ts looking like scope creep) came up repeatedly across rounds — confirmed false each time: those came from already-merged, unrelated origin/main commits (#499, #507, #525) pulled in by this branch's own merge commits, not new work. git diff origin/main...HEAD shows the true, clean #474-only scope.

Testing

cd middleware && npm install && npm run build && npm run lint && npm run typecheck && npm test — all green on the final merged branch: 4790/4794 pass, 0 fail, 4 pre-existing skips.

New/updated test files: middleware/test/orchestrator/pluginToolsReadyGate.test.ts, middleware/test/oauth/oauthReadinessTracker.test.ts, middleware/test/orchestrator/directLine.test.ts, middleware/test/cliBridge/toolDispatchService.test.ts, middleware/test/pluginContextStatus.test.ts.

Docs

• docs/middleware-agent-handoff.md — new "Plugin-Tool-Readiness-Gate (Non-authenticated Plugins are available to the orchestrator #474)" subsection.
• docs/CHANGELOG.md — full per-round [Unreleased] entry.
• docs/security-architecture.md intentionally not touched — this is an internal capability-gating mechanism, not a new auth boundary or credential flow; it doesn't fit that file's existing scope (credential storage, outbound-call proxying, plugin install trust boundary).

Closes #474

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}