Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .agents/skills/ai-video-gen/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Generate AI videos from text prompts. Supports multiple providers via four API p
| **Kling Official** | `KLING_API_KEY` | Kling official Classic, Turbo, and basic Omni video | `kling_official_video` |
| **Gemini API** | `GEMINI_API_KEY` / `GOOGLE_API_KEY` | Gemini Omni Flash (generation + conversational editing) | `gemini_omni_video` |
| **Pika API** | `PIKA_API_KEY` | Pika H3 Flex (native audio, first/last frame, up to 12 references, 4k upscale) | `pika_video` |
| **UberLLM** | `UBERLLM_API_KEY` | Seedance 2.0, Wan 2.7, Kling 3, Veo 3.1 Fast, LTX 2.3, Hailuo 02, Vidu Q3 (text, first frame, references) | `uberllm_video` |

**Iterative editing — Gemini Omni.** When the brief calls for *refining an existing clip* (add/remove objects, restyle, change lighting or on-screen text) rather than regenerating, Gemini Omni Flash is the only provider in the fleet with stateful multi-turn editing. See Layer 3 `gemini-omni` for the authoritative prompting guide (reference-image tags, timecode syntax, edit-prompt rules) before writing any prompt for it.

Expand All @@ -41,6 +42,7 @@ Use whichever configured gateway best matches the user's available providers and
- **Kling Official:** Set `KLING_API_KEY` to access Kling's official direct API via `provider="kling_official"`.
- **Gemini API:** Set `GEMINI_API_KEY` or `GOOGLE_API_KEY` to access Gemini Omni video generation and conversational editing.
- **Pika API:** Set `PIKA_API_KEY` to access Pika H3 Flex via `pika_video`. See Layer 3 `pika-h3-flex` for prompting, reference rules, and billing.
- **UberLLM:** Set `UBERLLM_API_KEY` to access several video model families via `uberllm_video`. See Layer 3 `uberllm-video` for model families, clip lengths, and billing.

Do not describe any gateway as the default or top choice without checking the registry and current task fit first.

Expand Down
86 changes: 86 additions & 0 deletions .agents/skills/uberllm-video/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
name: uberllm-video
description: |
Generate video through UberLLM (`uberllm_video`): Seedance 2.0, Wan 2.7, Kling 3 Standard, Veo 3.1 Fast, LTX 2.3 Fast, Hailuo 02 and Vidu Q3 Turbo on one UberLLM key, for text-to-video, image-to-video (first frame) and reference-to-video. Covers picking a model family, the fixed clip lengths each model takes, the free exact price check, billing, and resume after a polling timeout.
---

# UberLLM video

`uberllm_video` sends video jobs to UberLLM (`https://api.uberllm.dev`). One
`UBERLLM_API_KEY` reaches several video model families, and the same key and
prepaid balance also serve UberLLM's image, music and speech models, so a
production that needs more than video does not need another account.

## Route and billing

| Item | Value |
|------|-------|
| Tool | `uberllm_video` |
| Credential | `UBERLLM_API_KEY` (starts with `ull_`) from https://uberllm.dev/signup |
| Billing | Prepaid credit, pay as you go from $5. Failed, canceled and expired jobs cost nothing. A finished job is billed what the provider charged, never more than `data.max_cost_usd`. |
| Prices | Live per-model prices at https://uberllm.dev/models. Pinned per-second rates in `tools/provider_contracts/uberllm_video.json` are for preflight estimates only. |
| Cost reporting | `cost_usd` is the job's `usage.cost_usd`. `data.cost_status` is `settled` or `pending` (the exact charge usually arrives within 30 seconds of the file; until then the listed price is reported). |

Set `dry_run: true` to have UberLLM check the request (model, length, picture)
and return its exact quote without generating or charging. Set `max_cost_usd`
to have a job refused at no charge when it could cost more than that.

OpenMontage's default budget policy asks the user before any single action over
$0.50 and the first time a new paid tool is used.

## Choosing a model

Pass a family as `model`; the operation picks its text, image or reference
route. An exact UberLLM model id also works (see `get_info()["model_catalog"]`).

| Family | Operations | Good for |
|--------|------------|----------|
| `seedance-2.0` (default) | text, image, reference | Cinematic clips, the only family here with all three routes |
| `wan-2.7` | text, image, reference | Reference-guided clips at 720p/1080p, with a sound setting on reference-to-video |
| `kling-3-standard` | text, image | 3 to 15 second clips in 1 second steps, sound on unless `generate_audio: false` |
| `veo-3.1-fast` | text, image | 4, 6 or 8 second clips up to 4k, sound on unless `generate_audio: false` |
| `ltx-2.3-fast` | text, image | 1080p to 2160p clips of 6, 8 or 10 seconds |
| `hailuo-02-standard` | text, image | Low-cost 6 or 10 second clips |
| `vidu-q3-turbo` | text, image | Drafts: 1 to 16 seconds at 360p/540p for a few cents |

Draft on a low-cost family, then rerun the approved take on a premium one.

## Operations

| Operation | Inputs |
|-----------|--------|
| `text_to_video` | `prompt` |
| `image_to_video` | `prompt` + `reference_image_url` (https://) or `reference_image_path` (uploaded to UberLLM, kept 24 hours) |
| `reference_to_video` | `prompt` + up to 8 `reference_image_urls` / `reference_image_paths` |

Operation is inferred from the inputs when omitted. Pictures with
`operation: "text_to_video"` are an error, not a silent downgrade.

## Shared parameters

- `duration`: whole seconds. Each model makes fixed lengths (Seedance 2.0 text:
5 to 15; Veo 3.1 Fast: 4, 6, 8). Omitted: the model's shortest. A length the
model does not make is refused before anything is sent.
- `resolution`, `aspect_ratio`, `generate_audio`: sent only to models that have
the setting. Asking a model for a setting it lacks is refused with the list of
families that offer it, rather than ignored.
- `provider_params`: extra request fields, checked against the model's pinned
schema.

## Failure handling

- `insufficient_balance`: add credit at https://uberllm.dev/dashboard/credits.
Do not switch provider silently.
- A busy moment is retried by UberLLM itself (same job, same price ceiling);
`failed` means every try failed, and it costs nothing.
- Poll timeout: the result carries `data.resume_job`. Call `uberllm_video` again
with that object (and a longer `poll_timeout`) to keep waiting on the same job
instead of paying for a new one. Set `job_path` to persist the checkpoint.
- The tool never retries a paid submission on its own. Finished files are kept
about 3 hours, so the tool downloads them as soon as the job ends.

## Provenance

Every result records `data.request` (the payload sent, inline pictures
redacted), `data.job_id`, `data.usage`, `data.estimated_cost_usd`,
`data.max_cost_usd` and `data.resume_job`.
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,9 @@ LTX_API_KEY=
# Pika H3 Flex via the Pika API (text, first/last frame, reference-to-video with native audio).
# Create a key at https://dev.pika.art (API Keys). Prepaid, no free tier.
PIKA_API_KEY=
# UberLLM video (Seedance 2.0, Wan 2.7, Kling 3, Veo 3.1 Fast, LTX 2.3, Hailuo 02, Vidu Q3) on one key.
# Create a key at https://uberllm.dev/signup. Prepaid, pay as you go; the same key serves image, music and speech models.
UBERLLM_API_KEY=
# Runway Gen-4 (direct API, alternative to fal.ai routing).
RUNWAY_API_KEY=
# Volcengine Jimeng (即梦 AI) video generation via official API (HMAC-SHA256 V4 signing).
Expand Down
70 changes: 69 additions & 1 deletion docs/PROVIDERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ ARK_API_KEY= # API key body only; do not include the "Bearer " p

# VIDEO
PIKA_API_KEY= # Pika H3 Flex via the Pika API (native audio, references, 4k)
UBERLLM_API_KEY= # UberLLM video: Seedance 2.0, Wan 2.7, Kling 3, Veo 3.1 Fast, LTX 2.3, Hailuo 02, Vidu Q3
HEYGEN_API_KEY= # HeyGen avatar video gateway
RUNWAY_API_KEY= # Runway native + Seedance 2.5, Gemini Omni, MiniMax H3
SUNO_API_KEY= # Suno music generation
Expand Down Expand Up @@ -547,6 +548,72 @@ so 4k runs of 7 seconds or more will prompt the user.

---

### UberLLM — Video Generation

> **Several video model families on one key.** `uberllm_video` sends
> text-to-video, image-to-video (first frame) and reference-to-video jobs to
> UberLLM for Seedance 2.0, Wan 2.7, Kling 3 Standard, Veo 3.1 Fast, LTX 2.3
> Fast, Hailuo 02 Standard and Vidu Q3 Turbo. The same key and prepaid balance
> also serve UberLLM's image, music and speech models.

**Tools unlocked:** `uberllm_video`
**Env var:** `UBERLLM_API_KEY`
**Skill:** `.agents/skills/uberllm-video/SKILL.md`

#### Setup

1. Sign up at [uberllm.dev/signup](https://uberllm.dev/signup) and create a key (it starts with `ull_`).
2. Add credit under **Dashboard → Credits**. Usage is prepaid, pay as you go from $5.
3. Add to `.env`:
```bash
UBERLLM_API_KEY=ull_your-key
```

Optional: `UBERLLM_BASE_URL` overrides `https://api.uberllm.dev`.

#### What It's Best For

- **One key across families**: pick `seedance-2.0`, `wan-2.7`, `kling-3-standard`, `veo-3.1-fast`, `ltx-2.3-fast`, `hailuo-02-standard` or `vidu-q3-turbo` as `model`; the operation picks the family's text, image or reference route
- **Cheap drafts, premium finals**: draft on Vidu Q3 Turbo or Hailuo 02, then rerun the approved take on Seedance 2.0 or Veo 3.1 Fast
- **Free exact price check**: `dry_run: true` returns UberLLM's quote without generating or charging

#### API Notes

The UberLLM media API is asynchronous. Auth is `Authorization: Bearer <UBERLLM_API_KEY>`.

```text
# Submit (Idempotency-Key header on every submission)
POST https://api.uberllm.dev/v1/video/generations

# Poll until status is succeeded, failed, canceled or expired
GET https://api.uberllm.dev/v1/video/generations/{id}
```

- Each model makes fixed clip lengths (Seedance 2.0 text: 5–15 s; Veo 3.1 Fast: 4, 6, 8 s). A length, resolution or aspect ratio the model does not offer is refused before anything is sent, and asking for a setting the model lacks is an error rather than silently ignored.
- Request schemas (from `GET /v1/models/{model_id}`) and per-second rates (from the free `validate` quote) are pinned in `tools/provider_contracts/uberllm_video.json` and checked before any paid submission.
- Local first frames and reference pictures are uploaded to UberLLM's file store (`POST /v1/files`, kept 24 hours). `dry_run` sends them inline for the check instead.
- `max_cost_usd` has UberLLM refuse a job at no charge when it could cost more than that.
- On a polling timeout the result carries `resume_job`; pass it back to keep waiting on the same job instead of paying twice. The tool never retries a paid submission automatically.

#### Fallback Tools

If `uberllm_video` returns an error, the agent may retry with: `seedance_video`, `kling_video`, `veo_video`, `minimax_video`

#### Pricing

Billed per clip second; a job is never billed more than the `max_cost_usd`
held when it starts, and failed, canceled and expired jobs cost nothing. OpenMontage reports the job's `usage.cost_usd`
as the actual cost. Live per-model prices are at
[uberllm.dev/models](https://uberllm.dev/models); `get_info()["model_catalog"]`
lists the pinned rate per resolution used for preflight estimates. A 1 second
360p Vidu Q3 Turbo draft is about $0.03; a 5 second Seedance 2.0 text-to-video
clip is about $0.76, which OpenMontage's default budget policy asks the user to
approve.

> **Free tier:** None; usage is prepaid credit.

---

### Atlas Cloud — Image and Video Gateway

**Tools:** `atlas_image`, `atlas_video`
Expand Down Expand Up @@ -1570,6 +1637,7 @@ These tools require only FFmpeg or Python packages — no GPU, no API key.
| **Volcengine Ark** | `ARK_API_KEY` | `seedance_ark` | Pay-as-you-go |
| **MiniMax direct** | `MINIMAX_API_KEY` | `minimax_image`, `minimax_video` | Pay-as-you-go |
| **Pika** | `PIKA_API_KEY` | `pika_video` | Prepaid, per output second ($0.01–0.08/s) |
| **UberLLM** | `UBERLLM_API_KEY` | `uberllm_video` | Prepaid, per clip second (live prices at uberllm.dev/models) |
| **OpenAI** | `OPENAI_API_KEY` | `openai_tts`, `openai_image` | Paid only |
| **xAI** | `XAI_API_KEY` | `grok_image`, `grok_video` | Paid only |
| **Runway** | `RUNWAY_API_KEY` | `runway_video` | Free trial + paid |
Expand All @@ -1591,7 +1659,7 @@ How many providers cover each capability:
| Capability | Cloud Providers | Local Providers | Free Options |
|-----------|----------------|-----------------|--------------|
| **Image Generation** | FLUX, Kling Official, Grok, Google Imagen, GPT Image 2, Recraft | Local Diffusion | Pexels, Pixabay (stock) |
| **Video Generation** | Grok, Kling Official, fal.ai, Seedance via Volcengine Ark, Runway, Veo, Gemini Omni, Higgsfield, MiniMax, Pika H3 Flex, HeyGen, Tencent Hunyuan, ComfyUI Partner Nodes | WAN, Hunyuan, CogVideo, LTX, ComfyUI WAN, ComfyUI MiniMax H3 | Pexels, Pixabay (stock) |
| **Video Generation** | Grok, Kling Official, fal.ai, Seedance via Volcengine Ark, Runway, Veo, Gemini Omni, Higgsfield, MiniMax, Pika H3 Flex, UberLLM, HeyGen, Tencent Hunyuan, ComfyUI Partner Nodes | WAN, Hunyuan, CogVideo, LTX, ComfyUI WAN, ComfyUI MiniMax H3 | Pexels, Pixabay (stock) |
| **Text-to-Speech** | Azure AI Speech, ElevenLabs, fish.audio, Google TTS, Kling Official, OpenAI | Piper | Piper, Google free tier, ElevenLabs free tier, Azure free tier, fish.audio s2.1-pro-free |
| **Music Generation** | ElevenLabs, Suno, Google Lyria | — | ElevenLabs free tier |
| **Post-Production** | — | FFmpeg (compose, stitch, trim, mix, enhance, grade) | All free |
Expand Down
Loading