Skip to content

[Reconciliation][P1] Repair hosted lipsync, prevent silent creative-input loss and update skills #348

Open

Description

@sidneyswift

Parent workstream: #323
Epic: #318

Priority: P1 · Audit findings: SP-02, SP-11

Outcome and problem

The canonical content API reference and three customer-skill copies still prescribe reference_image_url, the old six-mode selector on POST /api/content/video, and string durations. Removal of that selector does not establish removal of every underlying capability: POST /api/content/create still accepts lipsync: true for the hosted template/content workflow. Image reference inputs now use optional image_urls; a supplied video duration must be an integer from 5 to 15 (default 5); retired reference fields can be stripped before a paid generation. The shared analyze-gate reference also claims removed Stop hooks enforce completion.

Lipsync correction and additional integration finding: The API validates and forwards lipsync: true to create-content. The current Tasks worker uploads the song audio and its Recoup wrapper sends audio_url to POST /api/content/video. That endpoint's current validator/buildVideoInput omit audio_url and the implementation always selects H3 Max. The worker then sets hasAudio from the requested lipsync flag, assuming generated video already contains the song. This is a source-level cross-repository integration mismatch; it does not establish the deployed worker/API combination or the outcome of a live preset run.

Scope and ownership

  • Owning repositories: recoupable/skills for workflow guidance, recoupable/api for validation and provider dispatch before billable generation, and recoupable/tasks for the existing hosted content worker. New background jobs must follow the current Vercel Workflow direction; do not create a second competing legacy task.
  • Prevent obsolete semantic inputs from being silently discarded. Reject unsupported legacy reference/mode inputs with actionable 400 errors, or explicitly map a safe backward-compatible alias before validation. Never silently turn reference editing, lipsync or extension into a different paid generation.
  • Preserve pinned house models, supported duration/resolution rules and credit gating; legacy compatibility must not restore arbitrary provider-model selection.
  • Update references/content-api.md and its copies in recoup-content-asset-pack, recoup-content-make-graphics, and recoup-content-make-video.
  • Preserve and repair the existing hosted lipsync template/content capability. Trace its selected preset/model and route audio to the appropriate supported, explicitly controlled audio-driven generation path with valid credit accounting. Do not retire lipsync solely because the low-level video endpoint no longer accepts the old mode selector.
  • Ensure final composition treats generated video as having audio only when actual output evidence supports that assumption; a requested lipsync flag is not proof of an audio stream or synchronization.
  • Adapt visualizer, first/last-frame and reformat guidance to their actual supported paths, and distinguish hosted lipsync workflow invocation from direct /content/video inputs.
  • Update canonical references/analyze-gate.md and all four copies to require explicit inspection without claiming installed hook enforcement.
  • Add focused example/schema checks for the request fields that drifted, while retaining existing vendored-byte parity.

Planning is tracked in Mono. Implementation and release PRs belong in the owning repositories under their current branch rules.

Acceptance criteria

  • Provider-free API regression tests prove that obsolete reference_image_url/images and unsupported video mode/audio/extension inputs are either safely translated or rejected before provider invocation and credit deduction.
  • Documented image examples use image_urls; a fixture proves the supplied reference URLs survive validation and reach the image input builder.
  • Documented video examples use integer duration 5–15 and the current H3 request fields; retired mode/string-duration examples are absent from active guidance.
  • A provider-free cross-repository regression fixture traces template/content lipsync=true through the worker to generation and proves the intended song audio survives validation and reaches the selected audio-capable provider branch, rather than silently generating an ordinary H3 video.
  • Final rendering does not infer the presence of audio from lipsync=true alone; missing output audio is detected before delivery, with explicit failure/recovery behavior.
  • Existing lipsync template/preset guidance is preserved and uses the verified hosted workflow. Extension behavior is separately reconciled; neither capability is declared removed solely because an old selector disappeared.
  • Before claiming the hosted lipsync workflow works live, an authorized bounded test records the deployed API/worker versions and verifies audible source audio plus visible mouth synchronization.
  • Every customer creative skill's completion guidance requires concrete inspection evidence and contains no claim that a Stop hook blocks delivery.
  • Fixtures validate examples against the current API schemas/builders without calling a paid provider. Five Skills gates, relevant tests and both package builds pass.
  • Release notes identify the corrected creative workflows; production generation is not claimed from fixture results alone.

Evidence and existing work

Baseline: October 10, 2026 audit of July 10–October 10 merges. Recheck current source and existing tracker comments before implementation. Source findings do not establish authenticated production behavior.

Lipsync workflow evidence:

Related existing work:

Continue or reconcile the linked implementation work; this ticket owns the stated audit correction and does not replace the original feature tracker.

Execution status — October 11, 2026

Underlying runtime and worker repairs are released through API #1028 and Tasks #157. Strict validation preserves compatible image reference aliases and rejects obsolete video semantic controls before spend. The dedicated bounded LTX audio-driven endpoint is deployed; existing preset/worker dispatch validates/trims selected audio before upload and checks output audio-stream presence. Ordinary video keeps song muxing. The reconciliation API deployment was READY at 2a882dd460ef6757af6d7b1b93bbc1fd8d0c6e86; exact worker commit 9f4ca0fdec32daad78457af3fb33dfa8c4410777 deployed as 20261011.1.

102 focused API checks and all 365 worker tests, including real ffmpeg/ffprobe fixtures, passed. Four invalid creative requests returned 400 in production without changing credits. Worker type checking retains eight existing errors versus nine at base, with no new signatures.

Current guidance/examples and explicit inspection requirements are released in Skills #183, Content #2, Marketing #149 and Docs #372. Content 0.3.1 and Skills 2026.1011.7 are published; no Stop hook is claimed to enforce delivery. Hosted lipsync remains a supported preset capability; an obsolete selector is not evidence it was removed.

Hosting work remains deferred by Sidney. October 12 readback now confirms the retained host publishes Research availability, its Markdown, research search Markdown and both LLM catalogs at HTTP 200, without research-provider branding. The human GET https://docs.recoupable.dev/mcp still returns 405; /mcp.md and the canonical Marketing human guide return 200. The October 11 restored-page 404s are historical rather than the current state of these checked pages. Final route/host compatibility and complete publication verification remain open.

Remaining acceptance: shared limiter availability under a real job, one bounded hosted selected-clip artifact with audible/faithful source audio and visible mouth alignment, plus installed-client/current retained-docs verification. Detecting an audio stream does not prove fidelity, audibility or synchronization. No successful production lipsync artifact is claimed.

Final October 11 production readback: API is READY at 925a57a704c7a9e32ee817e040b47a99ca144dd1 and Marketing at 8b36ab7677e3bd34c33e39f499518a301a2230c9. GitHub comparisons show each is one commit ahead of the reconciliation release, with no reconciled API-route or documentation-source changes: API comparison, Marketing comparison. Immutable release/provenance pins remain the reviewed reconciliation commits.

Activity

  1. changed the title [-][Reconciliation][P1] Update creative skills to the current image/video schemas and explicit QA checks[/-] [+][Reconciliation][P1] Prevent silent creative-input loss and update image/video skills[/+] on Oct 11, 2026
  2. changed the title [-][Reconciliation][P1] Prevent silent creative-input loss and update image/video skills[/-] [+][Reconciliation][P1] Repair hosted lipsync, prevent silent creative-input loss and update skills[/+] on Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocumentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions