SimpliGen Docs

An agent makes a music video

Give an AI agent a song and a cast, and watch it build a music video in the Video Studio, clip by clip, live.

With the SimpliGen connector (@simpligen/mcp 0.5.0 or newer), an agent such as Claude can build a whole music video in the Video Studio: it lays your song on the timeline, finds its tempo, puts clips on the bars, writes every shot, renders the takes and exports the film. Open the film in the studio while it works and you see every change happen. Anything it does undoes with Ctrl+Z, and the result is an ordinary film you keep editing by hand.

Before you start

SimpliGen must be running, the Video Studio installed, and an agent connected (see Connect an agent). Renders use your GPU, or cloud credits when the film renders in the cloud.

1. Ask for it

Give the agent the song, the people and the places, and say what should happen:

Make a music video for C:\Music\neon-rain.wav. Kai sings it on a rooftop at night; Mira walks a neon street in the second verse. 16:9, 1080p, rendered on this PC.

2. What the agent does

  1. The cast and the places. It reuses characters you have (list_characters) or makes them with create_character. A place it does not have yet it makes in three steps: generate a picture of it, wait_for_result, then create_location with that job.
  2. The film. create_film with the shape and resolution, then upload_file and add_track for the song (kind singing when it is vocals, so the singer's mouth follows them), and detect_tempo, so it can count in bars.
  3. The draft. One apply_film_plan lays out every clip on the bars with its shots, cast, location and whether it continues from the clip before. If anything is off (a name it misspelled, shots longer than their clip), every problem comes back at once and nothing changes until the plan is right.
  4. The renders. render_clips starts every clip that needs one; clips that continue from the clip before render one at a time. wait_for_renders waits until they are done.
  5. The review. It reads the timeline (get_film), fixes what needs fixing with update_clip, picks between takes with list_takes and use_take, and renders again where needed.
  6. The film. export_film stitches it with the song and answers the file.

3. Work on it together

Open the film in the Video Studio while the agent works. Every change appears live; press Ctrl+Z to undo one. You can edit at the same time: the agent always reads the film as it is now, your changes included. When it is done, the film is yours to keep editing like any other.

Good to know

  • Bars: after detect_tempo, clips and markers can be placed in bars ({ "bar": 5 }) and sized in bars ({ "bars": 2 }). Bar 1 is the song's first downbeat.
  • Continuing shots: a clip that continues starts on the last frame of the clip before, so its first shot keeps that framing.
  • Cloud renders: priced before anything starts and counted against the agent's spend cap.
  • Reference: every tool is in the MCP tool reference, every route in the Control API reference.