docker-agent/examples/gemini_image_output.yaml at main · docker/docker-agent · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
# Gemini image-output model: the model can generate an image directly as
# part of its reply, instead of only describing one.
#
# The `gemini-image` model below relies on the exact models.dev record's
# `Modalities.Output=image`. An explicit `output_capabilities.image` value can
# override catalogue data; unknown or unavailable catalogue metadata stays off.
# Capability is never guessed from the model name.
#
# On the models gateway, direct Gemini API, and Vertex AI, Docker Agent
# requests text AND image output for this image-capable model. It attempts
# to save each image in the workspace, record portable bytes, and display it
# in the TUI. Failures warn per item; unsupported graphics use path labels.
#
# Notes:
#   - Session-title/compaction omit image modalities and bypass the guard;
#     they do not explicitly force TEXT-only output. Ordinary requests
#     with custom tools or structured output are rejected locally before any
#     request is sent; Google Search, Maps, and code-execution built-ins remain
#     available. For custom tools, models.dev metadata lets the error say
#     whether the model lacks tool calls or only cannot combine them with
#     image output; unknown metadata keeps a conservative generic message.
#   - Naming: the model is instructed to honor an explicit prompt filename
#     in a naming marker; otherwise it chooses a meaningful name. A missing
#     marker falls back to a single parsed prompt filename only for a single
#     returned image, then to the provider name or a numbered generic name.
#     Multi-image replies use markers, provider names, or numbered names.
#     Extensions are corrected to match the returned data (the provider
#     decides the format, typically PNG), and existing files are never
#     overwritten — collisions get a dash suffix (logo-1.png).
#   - A text-only stop can trigger a nonfatal warning for phrases such as
#     "generate an image" in the last prompt. This is phrase matching, not
#     semantic intent detection: quoted/negated phrases can match and other
#     wording can be missed. It does not check output capability or track
#     a whole submission across tools, steering, stop hooks, or handoffs.
#     Terminal provider errors and current-agent structured-output config
#     skip the check; per-call overrides/reply content are not classified.
#   - Ordinary follow-ups send metadata placeholders, not generated bytes.
#     Attaching a saved image or reading it through a tool can send it again.
#
# Optional live trials require credentials, may be billed, and should use
# a disposable workspace and session store, not an active session database.
# Live Vertex image-generation validation is deferred.
# Try it out:
#   docker agent run examples/gemini_image_output.yaml \
#     "Generate an image of a red panda working at a terminal"
#   docker agent run examples/gemini_image_output.yaml \
#     "Generate an image of a lighthouse at sunset as assets/lighthouse.png"
#   docker agent run examples/gemini_image_output.yaml \
#     "Generate an image of a whale and save it as whale.jpg"
models:
  gemini-image:
    provider: google
    model: gemini-2.5-flash-image

agents:
  root:
    model: gemini-image
    description: Minimal agent for exercising Gemini native image-output generation.
    instruction: |
      When asked to draw or generate an image, do so directly using your
      native image-generation capability. Briefly describe what you made.