China Telecom's ShotPlan enables frame-level control for multi-shot AI video
Researchers from China Telecom and Harbin Institute of Technology have introduced ShotPlan, a framework for multi-shot cinematic video generation. The model utilizes learnable planning tokens and Fractional Temporal Rotary Position Embedding to maintain character consistency and enable precise frame-level camera control.
Key Takeaways
- Introduces learnable planning tokens that act as temporal anchors to guide transitions between shots in a single sequence.
- Utilizes Fractional Temporal Rotary Position Embedding (FRoPE) to achieve precise frame-level shot transitions, bypassing limitations of discrete latent timesteps.
- Maintains inter-shot consistency by generating entire narratives in one coherent process rather than independent segments.
- Supports diverse cinematic transitions including hard cuts, cross-fades, and localized camera motions like circular pans.
Why It Matters
ShotPlan addresses the 'continuity gap' that has long hampered AI video production, where maintaining visual identity across cuts usually requires manual post-production. By embedding shot planning directly into the diffusion transformer architecture, it moves generative video from a single-clip novelty toward a functional tool for narrative storytelling. This technical advancement reduces the computational overhead of keyframe-based methods while offering the granular timestamp control necessary for professional editors. For the industry, this signals a shift toward integrated cinematic workflows where AI models act as both the cinematographer and the editor, potentially lower costs for short-form drama and automated ad units. Watch for similar 'planning token' implementations in upcoming releases of larger foundation models like Sora or Kling.
Additional Context
The introduction of ShotPlan follows a significant push by Chinese state-owned enterprises into the generative AI space. In December 2024, China Telecom’s TeleAI institute launched its first in-house video generation model, ‘Xingchen,’ which claimed top rankings on VBench benchmarks for semantic consistency and scene relationships. Per TMTPost, the model was designed specifically to disrupt the high-growth short-drama and promotional video sectors in China, aiming to automate the entire pipeline from script to final edit. By mid-2026, this technology has matured into production-grade tools, with facilities like the Deqing AI virtual film base hosting over 30 projects and planning dozens of AI-generated short dramas, according to reporting from Xinhua in February 2026.
This trend coincides with a broader market shift toward cinematic-ready video output. By May 2026, leading models such as ByteDance's Seedance 2.0 and Kling 3.0 have moved beyond basic text-to-video prompts to support multi-shot narratives with native audio synchronization. Per AtlasCloud and Pinggy.io reporting from early 2026, the industry is transitioning away from 'tech demos' toward practical production tools that can output native 4K clips and hold character identity across multiple cuts. Research like ShotPlan is critical to this evolution, as professional creators increasingly demand the architectural flexibility to control specific camera movements and transition timing without relying on unpredictable prompt-based outcomes or expensive iterative generations.
Read full article at arxiv.org
Enjoy our coverage?
Add StreamingMeme as a preferred source on Google to see more of our streaming news at the top of your Search results.
Add as preferred source