Clipset
All articles
· 6 min read

MCP video editor: how Claude and Codex can drive a local video app

An MCP video editor is a video app that an AI agent can operate. Instead of you clicking through the interface, the app offers the agent a set of tools (read this recording, cut it into clips, add captions, render) over the Model Context Protocol, and the agent calls them while you watch the project change.

This piece explains how that works in practice, using Clipset‘s Agent Mode as the example since it’s the one I built and can describe precisely: how the connection is made, what the agent can call, and what stops it from doing things you didn’t ask for.

If you’re wondering whether this is useful at all before getting into the plumbing, start with can Claude or ChatGPT edit your videos?

The short version of MCP

MCP is an open protocol for giving AI apps tools. A server lists its tools, each with a name, a description and the arguments it takes. The AI app (the client) shows those tools to the model. When the model decides to use one, the client sends the call to the server and passes the result back.

For a video editor that means the model never touches pixels. It reads results (a transcript, a list of takes, a picture of a frame) and sends instructions. The app does the cutting, the compositing and the encoding, exactly as it would if you clicked the buttons yourself.

How the connection works

Clipset runs its MCP server inside the desktop app, on your own machine. It’s off until you turn it on in Settings › AI agent.

From there, Connect an app has a row for each AI app it can set up:

  • Claude Desktop. Clipset adds itself to Claude Desktop’s config file (after backing it up, and keeping any other servers you have). Quit and reopen Claude Desktop and the tools are there.
  • Claude Code. Clipset registers itself with the claude command for your user, so every Claude Code session can use it.
  • Codex (on Mac). The ChatGPT desktop app and the Codex app run local tools through Codex, so Clipset registers itself with Codex’s settings. Quit and reopen ChatGPT or Codex afterwards.

Under Other ways to connect there’s the server’s local address and a key, for any other MCP client that can talk to a local HTTP server. A new key cuts off the old one immediately.

A few details that matter for safety:

  • The server only listens on 127.0.0.1, so it’s reachable from your computer and nowhere else.
  • Every request needs the key.
  • Requests that come from a web page are refused, so a website you visit can’t drive the app.
  • The agent can only read files in the folders you allow in Settings (your home folder unless you narrow it).

The browser version of ChatGPT or Claude can’t use it, by design: a website can’t reach a server on your computer.

What tools the agent gets

About fifty tools, in groups. Real names, since that’s what you’ll see in the chat as the agent works.

Job Tools
See the project get_state, list_projects, open_project, new_project, list_combinations
Understand a recording analyze_media, get_frames, probe, propose_split
Cut and arrange split_recording, import_clips, add_section, move_clip, update_clip, reorder_sections
Text on screen overlay_presets, set_text_variants
Captions captions_setup, set_caption_style, caption_status, fix_captions
Sound analyze_music, set_audio_group, add_sound, synthesize_voiceover
Variety between videos set_transition, set_uniqueness, reshuffle
Check before rendering lint_project, render_preview_frame, render_one
Whole setup at once apply_recipe, export_recipe
Render estimate, generate, job_status, cancel_render
Take it back undo, redo

The interesting one is analyze_media. A model can’t watch a forty-minute recording, so this tool turns it into something a model can read: the words with timestamps, sentences, silences, scene cuts, loudness, and a contact sheet of frames. It also splits the speech into takes and groups the ones that repeat the same line, scoring each for filler words and pace so the agent can pick the cleanest. If you said “hook one” before a take or “scratch that” after it, those markers come through too. All of this runs locally, transcription included.

Recipes: the whole edit in one call

Calling forty tools one at a time is slow and leaves a messy undo history. So there’s apply_recipe: one description of the whole project, applied as a single undo step, with a dry run first that shows what it would make without changing anything.

A recipe for a typical batch looks something like this:

{
  "name": "Launch week",
  "sections": [
    {
      "name": "Hook",
      "clips": [{ "path": "/Users/you/Movies/Launch/hooks-take.mov", "best_of_group": true }],
      "overlays": { "per_clip": "first_sentence", "preset": "hook_top", "layer": { "place": "safe" } }
    },
    {
      "name": "Body",
      "clips": [
        { "path": "/Users/you/Movies/Launch/main-1.mov" },
        { "path": "/Users/you/Movies/Launch/main-2.mov" },
        { "path": "/Users/you/Movies/Launch/main-3.mov" }
      ]
    },
    {
      "name": "Ending",
      "clips": [{ "path": "/Users/you/Movies/Launch/endings-take.mov", "best_of_group": true }]
    }
  ],
  "music": [{ "from": "Body", "to": "Ending", "tracks": ["/Users/you/Music/track-1.mp3"], "duck": true }],
  "captions": { "on": true, "preset": "pop" },
  "uniqueness": { "enabled": true, "strength": "light" }
}

In words: take the best delivery of each hook line from one long recording, put each hook’s first sentence on screen where it doesn’t cover a face, three whole files as the main parts, the best ending takes, music under the body and ending that drops while someone talks, captions on, and small differences between videos.

You don’t write this. The agent does, after reading your recordings. export_recipe gives you the current project back as a recipe, so next week’s shoot can reuse this week’s setup with new files.

The guardrails

An agent that can render and edit needs limits that don’t depend on the model behaving. These are enforced by the app, not by instructions to the model:

  • No render without your yes. generate only works with a token from the latest estimate, which tells you how many videos the project makes. Change the project and the token expires. The agent has to show you the number and ask.
  • Undo for everything it does. Each change goes into an undo history (the undo tool, the Agent pill in the app’s top bar, or ⌘Z). If you’ve edited the project by hand since, its undo won’t throw away your work.
  • Nothing deleted. There’s no tool that deletes files. Removing a clip takes it off the board; the video file stays.
  • You see every change. Edits land on the board you have open, with a short glow where they happened, and a list of recent changes in the Agent pill.
  • Only your folders. Paths outside the allowed folders are refused, symlinks included.

Where the work happens

Rendering and transcription run on your computer, so the video files are never uploaded anywhere. What reaches the AI model is what the agent reads to decide: transcript text, still frames, the project layout. That goes to Anthropic or OpenAI the same way your typed messages do.

Two platform notes: placing text away from faces and on-screen text uses Apple’s Vision framework, so that check is Mac only. The voiceover tool uses the computer’s own text-to-speech voices on Mac and Windows.

Questions

Is this the same as an AI video generator? No. Generators make footage from a prompt. An MCP video editor edits footage you filmed. Different job.

Does it work with Cursor or other MCP clients? Any client that can connect to a local HTTP MCP server with a key should work through Other ways to connect. The one-click rows are for Claude Desktop, Claude Code and Codex.

What happens if the agent gets it wrong? You see it on the board, tell it what to change, or undo. Previews (render_preview_frame) let it check text placement before you see anything.

Does Clipset need to stay open? Yes. The MCP server lives in the app, and the app does the rendering.

For the practical side, the step-by-step from footage to finished clips is in one filming session to 50 clips with an AI agent. And the AI agent section on the home page plays the tool calls coming in against a live board, if you’d rather watch than read.

Try it on your own clips

Every hook × body × CTA combination, generated in minutes.

Clipset runs on your Mac or PC, never uploads your footage, and costs $99 $59.40 once.