Skip to content

feat(go): experimental agent middleware for sub-agent delegation and artifacts#5603

Merged
apascal07 merged 161 commits into
mainfrom
ap/go-session-flow-middleware
Jun 25, 2026
Merged

feat(go): experimental agent middleware for sub-agent delegation and artifacts#5603
apascal07 merged 161 commits into
mainfrom
ap/go-session-flow-middleware

Conversation

@apascal07

@apascal07 apascal07 commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Adds an experimental middleware package, plugins/middleware/exp, that brings two composable middlewares to the agent APIs in ai/exp: Agents lets one agent delegate to registered sub-agents through auto-generated per-agent tools, and Artifacts gives a model read/write tools over the session's named artifact collection. Both are ordinary generate middlewares (ai.NewMiddleware / ai.Hooks), attached with ai.WithUse on an agent's prompt, and they compose. To make them work, the Genkit instance is seeded into every agent's turn context automatically so middleware can resolve and run other registered actions via genkit.FromContext, and ai/exp gains the supporting surface: AgentRef (name a sub-agent, the agent analog of ai.ModelRef) and a State-agnostic ArtifactStore accessor for tools that touch artifacts without knowing the agent's State type.

Stacked on the agent API PR #4462; this PR targets ap/go-session-flow and its diff is just the middleware feature on top of that branch.

Examples

Orchestrator delegating to sub-agents

The Agents middleware turns each referenced agent into a delegate_to_<name> tool and injects a <sub-agents> listing into the system prompt. The orchestrator model delegates by calling the tool; the middleware runs the chosen sub-agent and returns its result.

import middlewarex "github.com/firebase/genkit/go/plugins/middleware/exp"

researcher := genkit.DefineAgent(g, "researcher",
    aix.InlinePrompt{
        ai.WithModelName("googleai/gemini-flash-latest"),
        ai.WithSystem("You are a research assistant. Summarize findings, then " +
            "call write_artifact to save them as a markdown artifact."),
        ai.WithUse(&middlewarex.Artifacts{}),
    },
    aix.WithDescription[any]("Researches a topic and summarizes well-sourced findings."),
)

engineer := genkit.DefineAgent(g, "engineer",
    aix.InlinePrompt{
        ai.WithModelName("googleai/gemini-flash-latest"),
        ai.WithSystem("You are an expert programmer. Write clean code, then call " +
            "write_artifact to save it as a file artifact."),
        ai.WithUse(&middlewarex.Artifacts{}),
    },
    aix.WithDescription[any]("Writes and explains code, producing file artifacts."),
)

orchestrator := genkit.DefineAgent(g, "orchestrator",
    aix.InlinePrompt{
        ai.WithModelName("googleai/gemini-flash-latest"),
        ai.WithSystem("You are a project coordinator. Delegate to the appropriate " +
            "sub-agent, then synthesize a final answer."),
        ai.WithUse(
            &middlewarex.Agents{
                // Descriptions are auto-discovered from each agent via Ref().
                Agents:           []aix.AgentRef{researcher.Ref(), engineer.Ref()},
                MaxDelegations:   5,
                HistoryLength:    4,
                ArtifactStrategy: middlewarex.ArtifactStrategySession,
            },
            &middlewarex.Artifacts{Readonly: true},
        ),
    },
    aix.WithSessionStore(store),
)

Sub-agents are referenced by AgentRef, either by name (aix.AgentRef{Name: "researcher"}) or captured from an agent value with agent.Ref() (which carries the agent's description into the system listing). The full version of this sample is in go/samples/basic-agents as the orchestrator agent.

Artifacts middleware

Artifacts gives the model read_artifact and write_artifact tools backed by the active session's artifacts. With no active agent session the tools degrade gracefully (empty listing), so the same prompt is safe to run standalone.

builder := genkit.DefineAgent(g, "builder",
    aix.InlinePrompt{
        ai.WithModelName("googleai/gemini-flash-latest"),
        ai.WithSystem("You are a code generator. Use write_artifact to create files."),
        ai.WithUse(&middlewarex.Artifacts{}),
    },
)

// Read-only: only the read_artifact tool is provided.
reviewer := aix.InlinePrompt{
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithUse(&middlewarex.Artifacts{Readonly: true}),
}

Composing delegation with shared artifacts

ArtifactStrategySession merges a sub-agent's artifacts into the parent session and keeps them out of the (potentially large) tool result. Pairing it with the Artifacts middleware on the orchestrator lets the coordinator read what its sub-agents produced before answering:

ai.WithUse(
    &middlewarex.Agents{
        Agents:           []aix.AgentRef{researcher.Ref(), engineer.Ref()},
        ArtifactStrategy: middlewarex.ArtifactStrategySession, // merge, don't inline
    },
    &middlewarex.Artifacts{Readonly: true}, // orchestrator reviews, doesn't produce
)

The default, ArtifactStrategyInline, instead includes artifact content in the delegation tool result so the orchestrator model sees it directly, and also merges it into the session.

Registering as a plugin (optional)

Using the middlewares via ai.WithUse needs no plugin. Register the plugin only to make them resolvable by name (e.g. for the Dev UI):

g := genkit.Init(ctx, genkit.WithPlugins(&googlegenai.GoogleAI{}, &middlewarex.Middleware{}))

API Reference

plugins/middleware/exp (experimental)

// Agents delegates to registered sub-agents through one generated tool per
// agent (delegate_to_<name>) plus a <sub-agents> system-prompt listing.
type Agents struct {
    Agents           []aix.AgentRef   // sub-agents available for delegation; >= 1 required
    ToolPrefix       *string          // tool-name prefix; nil => "delegate_to", &"" => bare names
    MaxDelegations   int              // cap per generate call; 0 = unlimited
    HistoryLength    int              // recent messages forwarded to client-managed sub-agents; 0 = task only
    ArtifactStrategy ArtifactStrategy // how sub-agent artifacts surface; default Inline
}

// ArtifactStrategy controls how a sub-agent's artifacts are surfaced.
type ArtifactStrategy string
const (
    ArtifactStrategyInline  ArtifactStrategy = "inline"  // content in tool result + merged into session (default)
    ArtifactStrategySession ArtifactStrategy = "session" // merged into session only; result names them
)

// Artifacts provides read_artifact / write_artifact tools over the active
// session's artifacts. With no active session the tools report so and list empty.
type Artifacts struct {
    Readonly bool // when true, only read_artifact is provided
}

// Middleware registers Agents and Artifacts as named Genkit middlewares.
// Only needed for name-based resolution (e.g. Dev UI); ai.WithUse does not need it.
type Middleware struct{}

ai/exp additions

// AgentRef names an agent for resolution against a registry (the agent analog
// of ai.ModelRef / ai.ToolRef). Resolves by Name, so the agent must be registered
// wherever the ref is consumed.
type AgentRef struct {
    Name        string // resolved as /agent/<Name>; required
    Description string // human-readable; filled by Agent.Ref; optional
}

// Ref derives an AgentRef from an agent value, capturing its name and description.
func (a *Agent[State]) Ref() AgentRef

// ArtifactStore is the State-agnostic view of a session's artifacts. Every Session
// satisfies it regardless of State, since artifact ops do not touch custom state.
type ArtifactStore interface {
    Artifacts() []*Artifact
    AddArtifacts(artifacts ...*Artifact)
}

// ArtifactStoreFromContext returns the active session's ArtifactStore, or nil if
// none. The accessor for middleware/tools that work with artifacts without knowing
// the session's State type (where SessionFromContext cannot help).
func ArtifactStoreFromContext(ctx context.Context) ArtifactStore

genkit additions

// FromContext returns the Genkit instance seeded into the context, or nil.
func FromContext(ctx context.Context) *Genkit

DefineAgent, DefinePromptAgent, and DefineCustomAgent seed their Genkit instance into each invocation's context automatically, so middleware running inside any agent's turns can resolve and run other registered actions with genkit.FromContext, just as genkit.Generate already seeds it. This is what lets the Agents middleware look up and run a sub-agent from inside a delegation tool.

Notes

  • The middlewares and their supporting ai/exp surface are experimental and may change in any minor release, tracking the agent APIs they build on.
  • History forwarding (HistoryLength) applies only to client-managed sub-agents (no session store); server-managed sub-agents receive only the task description, since their own store owns their history.
  • go/samples/basic-agents gains an orchestrator agent demonstrating delegation end to end; its .gitignore entry covers the compiled sample binary.

…pt/NamedPrompt

Replace the FromInline/FromPrompt agent-source constructors with three
clearer forms:

  - InlinePrompt(opts...)     defines the prompt inline (was FromInline)
  - SameNamedPrompt()         references the prompt named like the agent
  - NamedPrompt(name, input)  references any registered prompt by name,
                              rendered with an input supplied from code

NamedPrompt decouples the prompt's lookup name from the agent's name, so a
single prompt can back many agents with different inputs. The old
FromPrompt(defaultInput...) variadic-of-one is gone; per-turn input now
rides on NamedPrompt.

Updates the wrapper docs, both samples (chef's personality moves to the
.prompt frontmatter default), the README, and the tests, and adds
TestPromptAgent_NamedPromptSharedAcrossAgents covering the shared-prompt
path. Breaking change to the experimental ai/exp API.
Split the prompt-backed agent constructors so each has one clear job:

  - DefineAgent(r, name, prompt, opts...)  defines the prompt inline; the
                                           prompt is an InlinePrompt, a
                                           []ai.PromptOption slice passed
                                           positionally
  - DefinePromptAgent(r, name, opts...)    sources a prompt from the registry,
                                           defaulting to the agent's own name

DefinePromptAgent uses the same-named prompt by default; WithNamedPrompt(name,
input) points it at a different registered prompt rendered with a code-supplied
input, so one prompt can back many agents.

The prompt source is split across a compile-time-validated option set mirroring
ai/option.go: the shared options (WithSessionStore, WithStateTransform,
WithDescription) are AgentOptions valid on every constructor, while
WithNamedPrompt is a PromptAgentOption accepted only by DefinePromptAgent.
Passing it to DefineAgent or DefineCustomAgent fails to compile.

Making the inline prompt a required positional argument means an inline agent
cannot be defined without one. Removes the AgentSource abstraction and the
SameNamedPrompt/NamedPrompt sources.

Updates the wrapper docs, both samples, and the tests. Breaking change to the
experimental ai/exp API.
The "Load the Prompt from a File" path now uses DefinePromptAgent (default
same-named lookup) with WithNamedPrompt for shared prompts, and the inline
example uses the InlinePrompt slice literal.
…pans

Each runTurn-N span now records the committed session state at turn end as
its genkit:output, shaped as {state: <session state>}, and for
server-managed agents carries the turn-end snapshot's ID under
genkit:metadata:agent:snapshotId. The session ID stays on the root action
span as before.

The state is raw: StateTransform shapes only client-facing surfaces, not
telemetry or persisted state, so the span output matches the snapshot its
ID points to.

With the turn span output now derived from session state, the per-turn
chunk collection is gone: removed SessionRunner.collectTurnOutput,
chunkRouter.collectTurnChunks and its turnChunks/turnMu fields, and the
accumulation branch in applySideEffects (now artifact-only).
…rror

WithStreamTransform is the stream-side counterpart to WithStateTransform, rewriting each AgentStreamChunk on its way to the client. Both StateTransform and StreamTransform now return (value, error): a nil value omits the state or drops the chunk (wire-only), while a non-nil error (or a panic) fails the read or invocation closed with the transform's status preserved. Updates go/README.md and the genkit.DefineAgent option docs.
The abortSnapshot companion action's response carried only status, so a caller could not correlate the result with the snapshot it aborted. Add snapshotId (matching the abortSnapshot request) and populate it from the request.

Authored in the shared zod schema (genkit-tools/common/src/types/agent.ts) and regenerated across genkit-schema.json, the Go bindings (go/ai/exp/gen.go), and the Python typings (_typing.py); the Go doc and noomitempty come from schemas.config.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces experimental middleware for Genkit's Go agent APIs, specifically adding an Agents middleware for sub-agent delegation and an Artifacts middleware for session artifact access. It also introduces context decorators to propagate context values (like the Genkit instance) to agent turns, adds an AgentRef type for serializable agent references, and provides a state-agnostic ArtifactStore interface. The basic-agents sample has been updated to demonstrate an orchestrator agent coordinating specialized sub-agents. The review feedback highlights two potential nil pointer dereference panics: one in isClientManaged when handling a nil *aix.AgentMetadata pointer, and another in buildArtifactsListing if the slice of artifacts contains nil elements.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread go/plugins/middleware/exp/agents.go Outdated
Comment thread go/plugins/middleware/exp/artifacts.go
@apascal07 apascal07 requested a review from pavelgj June 24, 2026 02:07
Pull in the agent-source rework from ap/go-session-flow (InlinePrompt /
DefinePromptAgent replacing FromInline / FromPrompt / AgentSource, plus
WithStreamTransform) and reconcile it with the session-flow middleware work.

Conflict resolution:
  - ai/exp/option.go: keep both the new streamTransform option and the
    middleware's contextFunc option on agentOptions; retain WithContextFunc
    alongside WithNamedPrompt.
  - genkit/genkit.go: adopt the InlinePrompt signature for DefineAgent and the
    new DefinePromptAgent, and seed the Genkit instance into all three agent
    constructors (DefineAgent, DefinePromptAgent, DefineCustomAgent) so
    middleware can resolve actions via FromContext for prompt-backed agents too.
  - samples/basic-agents: keep the per-file agent layout and shared flashModel;
    port pirate/chef/orchestrator to InlinePrompt / DefinePromptAgent.
  - plugins/middleware/exp: port agents/artifacts docs and tests to InlinePrompt.
@apascal07 apascal07 force-pushed the ap/go-session-flow-middleware branch from a636fc4 to 1d7e550 Compare June 24, 2026 02:10

@pavelgj pavelgj left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a pass comparing against the JS @genkit-ai/middleware impl (agents.ts/artifacts.ts). Reads as a faithful, clean port and the test coverage is solid. A few small things inline.

One open question, not blocking: about half the Go plugins ship a README.md (alloydb, mcp, ollama, googlegenai, etc.) and the JS middleware package has one too, but plenty of Go plugins don't. The package doc comment in plugin.go is already good. Is it worth adding a short README.md here for parity, or do we consider the package doc sufficient? No strong opinion either way.

Comment thread go/plugins/middleware/exp/agents.go
Comment thread go/plugins/middleware/exp/agents.go Outdated
Comment thread go/plugins/middleware/exp/artifacts.go Outdated
Base automatically changed from ap/go-session-flow to main June 24, 2026 17:00
@apascal07 apascal07 requested a review from a team as a code owner June 24, 2026 17:00
- nil-guard typed-nil *AgentMetadata in isClientManaged
- document that unknown/absent agent metadata is treated as not
  client-managed (intentionally stricter than the JS middleware)
- align doc-example import alias with the sample (middlewarex)
- skip nil artifacts when building the artifacts listing
- count artifact size in runes, not bytes, to match the "chars" label
…ddleware

Resolve conflicts from main absorbing the agent foundation (#4462, #4797,
#4387, #5576) via separate squashed PRs:

- Adopt main's renames (AgentAbort{Request,Response}, Agent.Abort/AbortAction,
  abort routes) and its turn-context feature across go/ai/exp + schema.
- Preserve this branch's net-new middleware surface: ArtifactStore, AgentRef,
  WithContextFunc context seeding, and the orchestrator sample.

Per review direction, consolidate the agent constructors into the genkit/exp
(genkitx) surface that main migrated docs/samples/tests to:

- Move the genkit-instance seeding into genkitx.DefineAgent/DefinePromptAgent/
  DefineCustomAgent via a new genkitbridge.SeedContext hook, so middleware
  resolves sub-agents through the documented exp constructors.
- Remove the now-dead genkit.DefineAgent/DefinePromptAgent/DefineCustomAgent/
  ListAgents duplicates from package genkit.
- Reconcile the basic-agents sample to keep both the banker (interrupt/resume)
  and orchestrator (middleware delegation) demos on genkitx.
The root .gitignore carried Go-sample-specific entries that belong with the
module (a go/.gitignore already ignores the basic-agents binary):

- Drop go/**/.genkit: the repo-root bare `.genkit` rule already ignores
  .genkit dirs at any depth, so it was redundant.
- Drop the stale /go/custom-agent and /go/x-agent-interrupts binary ignores;
  those samples no longer exist.
- In go/.gitignore, generalize the binary comment and add /basic-agents-server
  alongside /basic-agents.

Snapshot artifacts stay covered by the root bare `.genkit` rule.
@github-actions github-actions Bot added the root label Jun 25, 2026
Tools created via NewTool/DefineTool (and the interruptible variants) wrap
ai.NewMultipartTool, whose function returns *MultipartToolResponse. The inner
ToolDef therefore advertised that envelope ({content, output, metadata}) as the
output schema instead of the real Out type, so the schema exposed to the model
and Dev UI was wrong. This made the experimental constructors strictly less
capable than ai.NewTool, which infers the output schema from Out.

Override Tool.Definition to set OutputSchema from the Out type parameter,
matching what ai.NewTool exposes. Genkit infers schemas with DoNotReference, so
the result is fully inlined and needs no registry resolution, making the
override equivalent whether or not the tool is registered. InterruptibleTool
embeds Tool, so it inherits the fix.

Add TestTool_OutputSchemaMatchesClassic, pinning the exp output schema to
ai.NewTool's for both the simple and interruptible constructors (nested struct
included to exercise schema inlining).
… API

The Agents and Artifacts middleware defined their tools with ai.NewTool, taking
an *ai.ToolContext. Switch them to aix.NewTool, the experimental constructor in
go/ai/exp, which takes a plain context.Context. All three tools (delegation,
read_artifact, write_artifact) only ever used the ToolContext as a context for
store and agent resolution, so the simpler signature is a clean fit and they
need none of the legacy ToolContext fields.

Behavior is unchanged: the tool results the model sees and the session artifact
operations are identical; only the construction API differs.
… WithContextFunc

The WithContextFunc agent option existed solely so genkit/exp's constructors
could seed the *genkit.Genkit into each agent turn, letting middleware reach it
via genkit.FromContext. Exposing that as a public option leaked an internal seam.

Move the seeding down into ai/exp's registry-level constructors: they now derive
the decorator from the registry they already receive, through a new internal
bridge hook (genkitbridge.SeedContextForRegistry) that the genkit package
installs and backs by reconstructing &Genkit{reg}. genkit/exp no longer injects
anything and WithContextFunc is removed from the public API; the contextFunc
field remains an unexported implementation detail.

The hook is nil-safe: an agent defined on a bare registry without the genkit
package linked carries no decorator. The genkit.Generate seeding path is
untouched, so middleware attached to a direct genkit.Generate call behaves as
before.
Add a "Delegate to Sub-Agents" example to the Agents section of the Go
README, showing the experimental Agents middleware (plugins/middleware/exp):
defining sub-agents with descriptions, wiring an orchestrator with
ai.WithUse, the key knobs (MaxDelegations, HistoryLength), and how it
composes with the Artifacts middleware. Mirrors the orchestrator in
samples/basic-agents.
@github-actions github-actions Bot added the docs Improvements or additions to documentation label Jun 25, 2026
@apascal07 apascal07 merged commit b477126 into main Jun 25, 2026
15 checks passed
@apascal07 apascal07 deleted the ap/go-session-flow-middleware branch June 25, 2026 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

config docs Improvements or additions to documentation go root

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants