Skip to content

Repository files navigation

Flow-Like YouTube Tools

Flow-Like WASM nodes for YouTube transcripts, metadata, search, channels, playlists, comments, thumbnails, subtitles, and URL utilities.

Prerequisites

  • Rust toolchain (1.88+)
  • WASM target: rustup target add wasm32-wasip2
  • curl and tar (already included with macOS and most Linux distributions)

Quick Start

mise run build
# Binary copied to: node.wasm

The mise setup task downloads the architecture-independent wasi-sysroot 25 release into the ignored .tools/ directory and verifies its pinned SHA-256 before extraction. For a manual build, set WASI_SYSROOT_PATH to an existing sysroot, or set WASI_SDK_PATH to wasi-sdk 25 or newer, then run:

cargo build --release --target wasm32-wasip2

Servo's target-specific SpiderMonkey archive is downloaded at build time; it is compiled into the component and is not a runtime service or sidecar. A separately verified archive can instead be supplied through MOZJS_ARCHIVE for build environments that require their own provenance-verification boundary.

Standalone YouTube Access

The transcript and summarize nodes run entirely inside the WASM component. On egress that YouTube still accepts anonymously they are zero-config: no sidecar, local service, provider URL, browser, or pre-minted token is required.

When YouTube challenges the ordinary InnerTube clients, the component fetches a fresh YouTube homepage and keeps that page's ytcfg/EVENT_ID paired with its ytAtN BotGuard challenge. It executes the challenge and current interpreter in an embedded SpiderMonkey runtime, then exchanges the resulting snapshot with GenerateIT. Because YouTube's homepage enrollment can return a valid challenge without the WebPO capability, the component makes up to three fresh, bounded attempts. It then tries YouTube's cookie-free Cobalt TV bootstrap up to three times, using each response-provided challenge key for that GenerateIT exchange. Only if those attempts fail does it make up to five fresh in-process /att/get attempts; that legacy enrollment is probabilistic and can be rejected for WEB sessions participating in YouTube's newer EVENT_ID binding experiment.

The integrity response is consumed by the still-live in-process minter to produce separate video-bound PLAYER and SUBS/caption proofs for the endpoints that enforce them. The player request reuses the enrolling page's exact visitor data, client version, and signature timestamp; visitor data is not used as the PLAYER content binding. There is no anonymous fallback-token shortcut, sidecar, external token provider, browser, or pre-minted token. The paired challenge/config and interpreter are obtained at runtime so deployments do not depend on a bundled, quickly stale YouTube script.

WebPO proves request integrity; it does not override YouTube's independent account/IP-reputation gate. If every first-party player client returns LOGIN_REQUIRED before exposing a signed caption track (as can happen on cloud datacenter egress), a valid PLAYER or SUBS token alone cannot recover the transcript.

For that case the transcript and summarize nodes accept an optional restricted egress relay through youtube_proxy_url and the sensitive youtube_proxy_token. The component remains standalone in the Lambda—there is no local process or sidecar—but all YouTube and BotGuard HTTP calls for that invocation leave through the remote relay. The included dependency-free Node.js service has an exact host/path/method allowlist and uses no YouTube account or cookies by default. See scripts/hetzner-proxy/README.md for the Hetzner installer, security model, and exact-video smoke test.

# On a Debian/Ubuntu Hetzner host after its DNS record points to the server:
sudo YOUTUBE_PROXY_DOMAIN=yt-proxy.example.com \
  ./scripts/hetzner-proxy/install.sh

Configure both node inputs together:

youtube_proxy_url   = https://yt-proxy.example.com/v1/fetch
youtube_proxy_token = <the installer token, supplied through a secret>

Hetzner is also a datacenter network, so its IP is not guaranteed to be accepted. Run the supplied smoke test before using it, then run the full cookie-free regression for the originally failing video through the deployed endpoint:

YOUTUBE_PROXY_URL=https://yt-proxy.example.com/v1/fetch \
YOUTUBE_PROXY_TOKEN='<secret>' \
./scripts/hetzner-proxy/smoke-test.sh

YOUTUBE_TRANSCRIPT_TEST_PROXY_URL=https://yt-proxy.example.com/v1/fetch \
YOUTUBE_TRANSCRIPT_TEST_PROXY_TOKEN='<secret>' \
cargo test --target "$(rustc -vV | awk '/host:/ {print $2}')" \
  --test integration \
  proxy_exact_reported_video_returns_caption_xml_without_cookies \
  -- --ignored --exact

As an account-backed alternative, both nodes still expose a sensitive youtube_cookies input. It accepts either a raw signed-in youtube.com Cookie header or a Netscape cookies.txt export. The component keeps the cookies inside the invocation, creates the required SAPISIDHASH headers, discovers the signed caption URL, and performs PO-token recovery itself. Cookies are account credentials: pass them through a Flow-Like secret variable, never commit or log them, and rotate the export when the YouTube session expires. The relay rejects account credentials by default; using cookies and the relay together requires the explicit server-side YOUTUBE_PROXY_ALLOW_ACCOUNT_AUTH=true opt-in.

Included Nodes

  • Video: transcript, metadata, thumbnails, caption languages, chapters, related videos, comments, oEmbed info, download
  • Search: YouTube search, trending videos
  • Channel: channel info, channel videos, channel playlists, RSS feed
  • Playlist: playlist videos
  • Utility: extract video/playlist/channel IDs, build URLs, thumbnail URLs, SRT, WebVTT
  • AI: transcript summarization with an LLM

Legal Use

Use this package only where you have the rights, permission, or another lawful basis to access and process the requested YouTube content. The download node is intended for lawful workflows such as content you own or are authorized to download; users are responsible for complying with YouTube's Terms of Service, API policies, and applicable copyright law.

About

Flow-Like WASM nodes for YouTube transcripts, metadata, search, channels, playlists, subtitles, thumbnails, comments, URL utilities, and AI transcript summaries.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages