An agent makes a music video
Give an AI agent a song and a cast, and watch it build a music video in the Video Studio, clip by clip, live.
With the SimpliGen connector (@simpligen/mcp 0.5.0 or newer), an agent such as Claude can build a whole music video in the Video Studio: it lays your song on the timeline, finds its tempo, puts clips on the bars, writes every shot, renders the takes and exports the film. Open the film in the studio while it works and you see every change happen. Anything it does undoes with Ctrl+Z, and the result is an ordinary film you keep editing by hand.
Before you start
SimpliGen must be running, the Video Studio installed, and an agent connected (see Connect an agent). Renders use your GPU, or cloud credits when the film renders in the cloud.
1. Ask for it
Give the agent the song, the people and the places, and say what should happen:
Make a music video for C:\Music\neon-rain.wav. Kai sings it on a rooftop at night; Mira walks a neon street in the second verse. 16:9, 1080p, rendered on this PC.
2. What the agent does
- The cast and the places. It reuses characters you have (
list_characters) or makes them withcreate_character. A place it does not have yet it makes in three steps:generatea picture of it,wait_for_result, thencreate_locationwith that job. - The film.
create_filmwith the shape and resolution, thenupload_fileandadd_trackfor the song (kindsingingwhen it is vocals, so the singer's mouth follows them), anddetect_tempo, so it can count in bars. - The draft. One
apply_film_planlays out every clip on the bars with its shots, cast, location and whether it continues from the clip before. If anything is off (a name it misspelled, shots longer than their clip), every problem comes back at once and nothing changes until the plan is right. - The renders.
render_clipsstarts every clip that needs one; clips that continue from the clip before render one at a time.wait_for_renderswaits until they are done. - The review. It reads the timeline (
get_film), fixes what needs fixing withupdate_clip, picks between takes withlist_takesanduse_take, and renders again where needed. - The film.
export_filmstitches it with the song and answers the file.
3. Work on it together
Open the film in the Video Studio while the agent works. Every change appears live; press Ctrl+Z to undo one. You can edit at the same time: the agent always reads the film as it is now, your changes included. When it is done, the film is yours to keep editing like any other.
Good to know
- Bars: after
detect_tempo, clips and markers can be placed in bars ({ "bar": 5 }) and sized in bars ({ "bars": 2 }). Bar 1 is the song's first downbeat. - Continuing shots: a clip that continues starts on the last frame of the clip before, so its first shot keeps that framing.
- Cloud renders: priced before anything starts and counted against the agent's spend cap.
- Reference: every tool is in the MCP tool reference, every route in the Control API reference.