claude-video

This is an Agent Skill for Claude Code, Codex, and other Agent Skills hosts that gives an AI agent the ability to "watch" a video. Point it at a YouTube link or a local file, and it returns timestamped frames plus a transcript, which the agent then uses to answer questions about what happens on screen and what's said. It's built for developers and power users who already run Claude Code or Codex locally (or in a compatible IDE extension) and want their agent to reason over video evidence instead of just text or screenshots.

The skill is written in Python and ships as a plugin/marketplace entry rather than a standalone app. It relies on external tools (ffmpeg, yt-dlp) for the local pipeline, and optionally calls Google's Gemini API for an end-to-end "hand the whole video to a model" path, or Groq/OpenAI Whisper APIs for cloud transcription. Native video captions are used first when available, which avoids any transcription step entirely.

Key features

• Dual engine design: auto mode picks Gemini when an API key is present, otherwise falls back to a fully local pipeline (ffmpeg + yt-dlp + optional Whisper).
• Caption-first transcription: reads native captions on URLs before trying any speech-to-text model, saving time and API calls.
• Pluggable transcription backends: choose whisperx (local, no API key, separate managed Python 3.12 environment), groq, openai, or none for visual-only analysis.
• Timestamped output: frames and transcript segments carry timestamps so the agent can cite exact moments in a video.
• URL and local file support: works with YouTube and other yt-dlp-supported sources, or with video files already on disk.
• Engine isolation: Watch never silently switches engines; a Gemini failure is reported with its error category instead of auto-falling-back to local processing.
• Config via dotenv: API keys (GEMINI_API_KEY, GROQ_API_KEY, OPENAI_API_KEY) resolve from environment variables or ~/.config/watch/.env, with per-provider lookup order and no automatic key-borrowing.

Ideal use cases

This fits well when you're using Claude Code (desktop Code session, VS Code extension, or terminal CLI) or Codex and want the agent to answer questions about video content: summarizing a tutorial, checking what happens at a specific timestamp, extracting dialogue from a clip, or reviewing a screen recording. It also works for private local video files that shouldn't go through a cloud API, since the local engine with --engine local never contacts Google.

It is not a fit for Claude Chat, Cowork, or other browser-only agent surfaces; the README is explicit that these aren't supported, partly because cloud sandboxes often lack ffmpeg/yt-dlp or block outbound downloads from hosting providers, and because API keys don't persist across Cowork tasks. It also isn't a general-purpose video editor or transcription service; it's an agent capability add-on, so you need an existing Claude Code or Codex setup to use it at all. If your videos are long and you need guaranteed speech transcription without API keys, you'll want to budget disk (3GB+) and RAM (8GB+) for the local WhisperX backend, since that's the only key-free speech option.

Installation

Installation depends on which agent host you use. For Claude Code in a terminal:

claude plugin marketplace add bradautomates/claude-video
claude plugin install watch@claude-video

For Claude Desktop, add the marketplace through Customize → Plugins → Add → Add from a repository, pasting https://github.com/bradautomates/claude-video, then install the watch plugin and start a Code session.

For Codex, paste this into the message box:

Use $skill-installer to install the watch skill from:
https://github.com/bradautomates/claude-video/tree/main/skills/watch

For other Agent Skills hosts (Cursor, Copilot, etc.), install Node.js if needed, then run:

npx skills add bradautomates/claude-video -g --skill watch

After installation, Watch itself needs Python 3.10+, FFmpeg/ffprobe, and current yt-dlp on the machine (or cloud sandbox) where the agent executes code. Install these with your package manager, for example on macOS: brew install python ffmpeg yt-dlp. YouTube downloads additionally need a JavaScript runtime like Deno. If you want local speech transcription, have the agent run the bundled setup.py --install-whisperx, which provisions its own Python 3.12 environment and WhisperX without touching your base Python install.