Structured JSON & Timed Beat Prompting for AI Video Generators

1. Separate a production brief from a provider request

A provider's request format is defined by that provider's current documentation. It may accept a prompt string, reference images, a duration, or specific camera controls; another provider may expose different fields. The JSON below is your own intermediate brief. Do not post it to an arbitrary generation endpoint and assume every key will be interpreted. Instead, map supported controls deliberately and convert unsupported fields into concise prompt language or editor notes. Keep credentials and provider-specific authentication out of the downloadable creative brief.

This distinction also improves portability. If a model changes its available durations, your original storyboard can remain intact while the generation plan changes to shorter clips assembled in an editor. If a model does not accept separate dialogue, use the field to guide a separate voiceover recording rather than claiming synchronized speech support. A planning contract should state what the finished piece needs. A provider request should state only what that particular system can actually receive. Treat the mapping between them as a reviewed production step.

2. Put stable identity outside the changing shot instructions

The characterLock object contains an ID, visual traits, and props. Use the ID as a label for humans and production records, not as a magical reference that a generator knows from an earlier session. Visual traits should be observable: circular white head, dot eyes, cobalt hoodie, narrow black limbs, and white shoes. Avoid vague descriptions such as friendly and cinematic when they are supposed to identify the character. Those adjectives may influence mood but do not specify the silhouette or costume that must match across cuts.

Props need equally explicit rules. State which hand holds the mug, whether it changes scale, and whether it remains visible. Put a beat-specific reaction in the timeline rather than changing the locked identity description for that beat. If the story genuinely changes the prop state, distinguish state from identity: the same red mug can become empty without becoming a different cup. When generating separate shots, repeat the complete identity and initial prop state in each standalone request, even though your master brief stores them once for clarity.

3. Use explicit beat boundaries and one primary action per beat

The example describes a ten-second cup demonstration. Each timeline entry has a numerical start and end, a camera description, a lighting description, a visible action, and a dialogue string. The first beat starts at zero and the last ends at ten. Adjacent boundaries meet without accidental gaps. These are planning constraints to inspect in your own brief; the current builder does not turn them into rendered animation or guarantee that a downstream generator follows the exact timing. The final edit must establish the actual boundaries.

Keep action fields concrete enough to judge in a preview. The character realizes the importance of boundaries is an interpretation; the liquid stops at a black fill mark is visible evidence. A model can produce a plausible smile while failing to show the lesson. Write the visible endpoint first, then choose the smallest motion that reaches it. For dialogue, preserve the exact intended spoken sentence once. Repeating alternative versions in several fields can create ambiguity during a manual handoff or encourage a model to include unwanted extra text.

{
  "format": {
    "aspectRatio": "9:16 vertical",
    "resolution": "1080x1920",
    "durationSeconds": 10
  },
  "artStyle": "2D Stick Figure Animation",
  "characterLock": {
    "id": "HOST-01",
    "visualTraits": "White circular head, dot eyes, blue hoodie, black limbs",
    "props": "Same red mug in right hand; fixed size"
  },
  "timeline": [
    {
      "label": "Hook",
      "start": 0,
      "end": 3,
      "camera": "Close-up",
      "lighting": "Flat 2D Clean",
      "action": "Liquid approaches the mug rim; pouring continues.",
      "dialogue": "You keep adding.",
      "estimatedVoiceoverSeconds": 1.2
    },
    {
      "label": "Consequence",
      "start": 3,
      "end": 7,
      "camera": "Medium Shot",
      "lighting": "Flat 2D Clean",
      "action": "Liquid spills. The left hand indicates the fill mark.",
      "dialogue": "Then wonder why it spills.",
      "estimatedVoiceoverSeconds": 2
    },
    {
      "label": "Result",
      "start": 7,
      "end": 10,
      "camera": "Close-up",
      "lighting": "Flat 2D Clean",
      "action": "Pouring stops at the mark; hold the result.",
      "dialogue": "Set a limit before you start.",
      "estimatedVoiceoverSeconds": 2.4
    }
  ],
  "continuityRules": [
    "Preserve silhouette, outfit, prop identity, and prop hand.",
    "No extra characters or decorative text.",
    "Keep essential action clear of the selected UI overlay."
  ]
}

4. Validate syntax, structure, and creative feasibility separately

Syntax validation asks whether the text is legal JSON: double-quoted keys, balanced brackets, escaped line breaks inside strings, and no trailing commas. Structure validation asks whether the fields and value types match your production format. Creative validation asks whether the story can actually be shown in the available time. Passing the first test says nothing about the other two. A timeline with an end before its start is legal JSON, and a three-second action containing six independent gestures can be structurally valid but impossible to read clearly.

Review the numerical timeline in order. Confirm finite values, positive beat lengths, no unintended overlap, and a final boundary consistent with the total duration. Then read the action and dialogue together. At the builder's planning rate of 2.5 words per second, a twenty-word line needs eight seconds, so it does not sensibly belong in a three-second beat. The warning is useful, but a line below the threshold still needs a scratch recording. Names, numbers, pauses, and performance can all alter its spoken length.

5. Adapt fields to a generator without adding unsupported promises

Begin with the character reference and the shot's initial state, then state the motion, camera, lighting, and required endpoint in plain language. Use the model's actual reference-image or duration controls only when they exist. Do not assume a JSON property called seed, character_id, or lip_sync has an effect merely because you add it. Unsupported keys can be ignored or rejected. Keep a provider-specific note outside the article's generic example and verify the current interface before turning your brief into a real request.

Avoid overloading one generation with incompatible visual directions. A locked frame cannot simultaneously make a large orbit, and a flat vector drawing does not need photorealistic lens bokeh to become readable. If a camera move is essential, describe its direction and stopping point while keeping the subject's identity and screen direction stable. If an exact match is essential, consider separate controlled artwork or compositing instead of relying entirely on stochastic generation. Structured prompting makes decisions inspectable; it does not remove the need for an appropriate production method.

6. Version the brief, references, and output together

Give each experiment a stable local version label and record which source brief produced it. Keep the model name, model version where exposed, reference assets, requested duration, and relevant settings in a production log. Those details do not all belong inside the creative prompt, but they matter when a promising result needs to be reproduced or compared. If you only save the final text string, you may lose the reference image or configuration that explains why one shot preserved the face and another did not.

Change one meaningful variable per comparison. For example, keep identity, action, and duration fixed while replacing a wide shot with a medium shot. Inspect whether the hand and mug remain legible at phone size. If you change five fields at once, the better output cannot tell you which decision mattered. Use a text diff between JSON versions to catch accidental changes to a locked trait. A new camera instruction should not silently replace the character's hoodie color or move the mug to the other hand.

7. Treat the exported file as a handoff checklist

Before handing the brief to an animator or editor, remove abandoned alternatives and confirm that every referenced prop exists in the asset set. Include the final dialogue, timing intent, continuity rules, and delivery frame. Explain which values are estimates and which are hard requirements. For example, 1080 by 1920 may be the delivery specification while a three-second opening is an editable target. That distinction lets a collaborator adjust the performance without accidentally changing a requirement that the project actually depends on.

After generation, compare the output with the brief shot by shot. Check identity, prop hand, action endpoint, screen direction, and room for captions. Do not mark a shot successful simply because it looks polished. A technically attractive frame can still fail the demonstration or hide the object the narration names. Keep approved deviations in the next brief version so the document remains an accurate record rather than an obsolete wish list. The benefit of structured prompting is this repeatable review loop, not the appearance of technical complexity.

Frequently asked questions