In a post published on the Google Research blog by research scientists Yale Song and Yiwen Song, a unified multi-agent framework was introduced for the autonomous generation of temporally consistent, long-form video narratives. The researchers noted that while diffusion models can generate high-fidelity video clips in seconds, transforming them into coherent, long-form storytelling engines remains a significant challenge. Most existing agentic pipelines automate the process via chained modules but suffer from semantic drift—such as subtle shifts in character attire or scenery across shots—and cascading failures, where an upstream asset defect corrupts downstream video generation.
According to the researchers, these failures stem from independent, handcrafted prompt engineering, and because early errors propagate and impair long-horizon consistency, the process often requires extensive manual intervention. Structurally, this reflects the credit assignment problem, as terminal failures are difficult to trace back to specific prompts. In addition, existing methods suffer from feature drift, where entities and environments gradually change unintentionally, or content collapse, where narratives fail to progress meaningfully. Google's research presents a suite of four frameworks—Co-Director, CANVAS, A²RD, and VQQA—designed to translate high-level human creative direction into execution by automating repetitive orchestration tasks, from multi-model prompting and shot chaining to closed-loop visual refinement.
Orchestrating Creative Intent with AI Video Co-Director
To ensure semantic coherence across an entire video, the researchers introduce the AI video co-director, a hierarchical multi-agent framework that formulates video storytelling as a global optimization problem. Rather than relying on rigid, linear prompt chains, the framework introduces hierarchical parameterization: a multi-armed bandit (MAB) algorithm globally identifies promising creative directions, balancing the exploration of novel narrative strategies with the exploitation of effective creative configurations. The system samples abstract creative trajectories—such as combining an informational strategy with a vignette narrative mode and a defined aesthetic archetype—and dynamically injects them into the sub-agents' prompts.
Orchestration is executed across two interconnected loops: strategic steering and multi-stage production. The Orchestrator Agent selects a creative configuration across three dimensions: creative strategy (intent), narrative mode (story structure), and aesthetic archetype (visual tone and cinematography). The Pre-Production Agent synthesizes a scene-by-scene storyline brief and visual assets into a unified storyboard. The Production Agent translates the storyboard into media using specialized sub-agents: a Keyframe Agent anchoring character and scene visuals, a Video Agent adding motion, and an Audio Agent adding matching voiceover and score. A multimodal large language model judge (MLLM Judge) evaluates the final cut and feeds a factored reward signal back to the MAB for refinement and optimization in subsequent generation loops. Because the framework operates as an orchestration layer on top of Gemini and Veo, image, video, and audio outputs inherit safety mechanisms such as SynthID watermarking, and additional safety classifiers can be applied across the final video.
Visual Consistency Planning with CANVAS
To resolve broken consistency and character shifts between consecutive and non-consecutive shots, Google introduces CANVAS (Continuity-Aware Narratives via Visual Agentic Storyboarding). This framework operates as an orchestration layer on top of Gemini and maintains structured representations of characters, locations, and object states as the narrative evolves, relying on persistent visual memory. By retrieving visual anchors from memory or initializing new ones when needed, the system is designed to ensure smooth transitions and preserve character identity and the spatial structure of environments when revisited.
In a test presented in the post on a museum heist sequence from HardContinuityBench, CANVAS was compared against the Gemini-3.1-Pro baseline model as well as an alternative multi-agent framework named AutoStudio. As shown in the research, direct generation using Gemini-3.1-Pro exhibited object inconsistency and background drift where the room layout shifted. AutoStudio exhibited character drift where the thief's cap disappeared across cuts. In contrast, CANVAS maintained the consistency of characters, spatial geometry, and object states throughout the narrative thanks to its persistent visual memory.
Managing Temporal Dynamics with A²RD
To translate storyboards into minutes-long video, the A²RD architecture was developed for agentic autoregressive video generation. A²RD operates via segment-by-segment generation supported by a multimodal video memory that tracks segment contexts and dynamics. For each segment, the agent operates in a retrieve-synthesize-refine-update loop.
A key part of this loop is the ability to adaptively determine the segment generation mode: switching between extrapolation to allow for natural narrative progression, and interpolation that anchors segments to existing entities and environments. In an experiment presented by the researchers on a ten-minute film, the system continuously queried its multimodal video memory to maintain character identity, costume details, and structural geometry across multi-minute temporal gaps.
Closed-Loop Refinement with VQQA and Evaluation Findings
To autonomously identify and correct visual defects without relying on a white-box approach to the model, the VQQA (Video Quality Question Answering) framework was developed. The framework generates visual questions tailored to the specific prompt and uses vision-language model (VLM) critiques as semantic gradients—natural language directional feedback that guides iterative refinement, analogous to numerical gradients. VQQA operates as a black-box prompt optimizer: rather than modifying pixels directly, it refines the text prompt to correct compositional defects like attribute binding errors or character inconsistencies, guiding the video generator to sample a new path in latent space. To prevent semantic drift during enhancement, a Global Selection mechanism evaluates every video generated along the optimization trajectory against the original prompt and selects the highest-scoring candidate.
To evaluate the frameworks, the researchers developed three benchmarks: the GenAD-Bench benchmark featuring 50 fictional brands with four products each across 400 scenarios; the HardContinuityBench benchmark testing spatial and environmental continuity with large disappearance gaps; and the LVBench-C benchmark comprising 120 text scenarios where critical assets disappear for at least 10 segments before returning. In evaluations, AI video co-director achieved a peak quality score of 81.4 on GenAD-Bench, while CANVAS demonstrated consistency improvements on ST-Bench and HardContinuityBench. The A²RD architecture reduced layout drift on VBench-Long and LVBench-C, and VQQA achieved quality improvements on T2V-CompBench, VBench2, and VBench-I2V. The researchers noted that their goal is not to replace human storytellers, but to assist them by abstracting away the complexities of world-state tracking and long-horizon consistency.