A video-editing agent receives a modest request: move a voiceover to the three-second mark. The tool runs, but the response never arrives. The agent retries. Meanwhile, the user trims a shot in the editor. Should the second call run? Which project version should it modify? What happens to the captions linked to that voiceover?
These are state-management questions. A more detailed prompt cannot, by itself, make an editing operation atomic or prevent a stale plan from overwriting newer work.
I maintain Timeline Studio, an open-ade me think about editing interfaces in terms of revisions, operation identities, semantic differences, and output receipts. This article describes that design and its current limits
The second request changes the architecture
Generating a finished video can be a bounded task: chooseit that video requires a durable representation of the decisions behind it. A request such as “restore the removed shot after the second scene” needsmed audio and captions
Timeline Studio preserves those decisions in an editable .timeline archive. The local workflow stores project.json alongside embedded media. Asset import probes necessary metadata and records SHA-256 integrity information. The rendered video is an output of the project, while the project remains available for subsequent edits.
An editing command needs an identity
Consider moving an existing narration clip. The structured plan identifies the project, its base revision, the target clip, an operation type, and an operation ID. It also declares a new output path. The command engine validates the operation against the actual project rather than accepting a plausible-looking object as sufficient evidence.
The operation ID makes retries tractable. If an action was already applied, repeating that identity should not perform the edit again. This matters particularly for insertion: a lost response must not turn one intended shot into two copies. The agent must preserve the operation identity when retrying the same intent.
Idempotency does not decide whether an editorial choice is good. It provides a narrower guarantee: a repeated request does not accidentally multiply its effects.
Optimistic concurrency belongs in creative tools
The plan also carries baseRevision. If the agent read revision 12 and the user subsequently edited the project, a stale plan containing new operations is rejected with a revision conflict. The agent then needs to inspect the current state and reconsider its changes.
That is preferable to assuming the agent has exclusive control of an interactive editor. A person may be adjusting a clip while the agent is analyzing media. The interface should preserve that work instead of treating it as an inconvenience.
This mechanism protects state transitions, not artistic intent. Two valid plans can still disagree creatively. The system can expose the conflict; deciding the desired edit remains a planning or user decision.
A dry run should describe meaningful changes
A generic JSON diff can be difficult to review. Reordering one clip may change several serialized fields even when the editorial intention is simple. The useful question is: which clips were added, removed, reordered, or modified, and did the sequence duration change?
Timeline Studio exposes project.diff as a project-aware dry run. It runs the same command registry and reducers used by application, without writing an archive. A separate validator checks transport shape, finite time values, and duplicate operation IDs. The semantic dry run checks the plan against actual project state and reports predicted changes.
The intended sequence is inspect, plan, validate, diff, apply, and inspect again. This separates a syntactically valid request from an applicable edit, and an applicable edit from a verified result.
A transaction has to include the editing relationships
Moving a narration clip may require moving its still-linked captions by the same delta. Main visuals form a contiguous sequence, while overlays, narration, and captions carry explicit timing. Ripple mode, links, and track locks affect which objects should move after a duration change.
Those relationships belong in shared application rules. Asking the model to remember every invariant on each call makes correctness dependent on context retention. A batch should pass validation before it is committed; a failed local batch should not produce a partially edited output archive. Local CLI and MCP writes require a new output path and reject overwriting the input or an existing output.
Video introduces a second consistency problem: time
A model can correctly locate an action at ignores trimming, placement, or retiming. Analysis should retaintiming rules. This mapping also affects keyframes, thumbnails, and effects
Keep transports thin
The project exposes local archive editing through a CLI and a local STDIO MCP adapter. A browser WebMCP route, when available in the host, operates on the open editor and integrates with its normal history. These routes should converge on shared editing logic rather than maintain separate interpretations of a trim or move.
The Skill serves a different purpose: it describes how the agent inspects material, plans a cut, chooses the available execution path, and verifies its work. MCP supplies callable operations. A workflow description and a command contract complement each other, but neither replaces the other.
A valid transaction is not a finished video
The local project.render path currently supports the documented portable Visuals, Voiceover, and Music subset. It produces H.264/AAC MP4 through FFmpeg and checks dimensions, duration, and audio-track presence with ffprobe. Unsupported composition features are rejected rather than silently omitted; richer compositions need the browser renderer.
That boundary is deliberate. An apparently successful export that loses captions or overlays would be harder to trust than an explicit unsupported-feature error. Even a structurally correct project still needs reopening, media-resolution checks, playback around cuts, and actual listening. Container metadata cannot establish that narration sounds correct.
What I would reuse in another agent application
The useful abstraction is a reviewable state transition: observe current state, declare intent with preconditions, preview its consequences, commit once, and inspect the result. Documents, spreadsheets, design tools, and scene editors can benefit from the same separation. The model supplies judgment; the application supplies enforceable rules and observable outcomes.
The implementation is available in the Timeline Studio repository. The shared engine lives in src/lib/projectCommandEngine.js, the local runner in scripts/timeline-command.mjs, and the adapter in skills/edit-timeline-studio/mcp/server.mjs. Current commands and limitations are documented under skills/edit-timeline-studio/docs/command-reference.md.