A Lightweight Framework to Tackle Multi-Shot Generation Challenges
Recently, the field of video generation has been advancing from single-shot generation to multi-shot narratives. However, mainstream solutions rely on models with massive parameter counts, raising the technical barrier. To address this challenge, researchers from Dalian University of Technology, Kuaishou’s Kling team, and The Chinese University of Hong Kong have jointly introduced MultiShotMaster, a highly efficient and controllable framework for multi-shot video generation. This research has been accepted by the top-tier computer vision conference CVPR 2026 (Paper ID: 2512.03041). The training and inference code for models based on 1.3B and 14B parameters has been open-sourced, aiming to lower the development cost of multi-shot video creation.
Core Technical Innovations Based on Rotary Position Embedding
The core of MultiShotMaster lies in a clever modification of existing video generation architectures, rather than introducing extra parameters. Its technical innovations are mainly reflected in two aspects:
- Multi-Shot Narrative RoPE: To solve the problems of abrupt content changes and narrative disorder during shot transitions, the researchers proposed applying a phase offset at shot boundaries on top of the original 3D Rotary Position Embedding (RoPE). This method explicitly marks shot transitions and maintains narrative timing, enabling the model to accurately recognize and generate videos with user-specified numbers and durations of shots.
- Spatiotemporal Position-Aware RoPE: To achieve fine-grained control over elements like reference images and subject motion, the framework designed a spatiotemporal position-aware RoPE. By resampling the RoPE for specific spatiotemporal regions and assigning it to the tokens of the reference image, the model can precisely inject a specific subject or background into any spatiotemporal position in the video. This mechanism is also applicable for controlling the motion trajectory of the same subject.
Furthermore, the framework employs a general-to-specific prompt structure and a Multi-shot-Multi-subject Attention Mask to ensure accurate execution of shot-level instructions and prevent unnecessary information leakage.
Automated Data Construction and Performance Validation
To support model training, the research team established an automated pipeline for multi-shot video data annotation. This pipeline utilizes TransNet V2 for shot segmentation, uses Gemini-2.5-Flash to generate global and shot-level descriptions, and integrates tools like YOLOv11 and SAM to complete cross-shot subject tracking and segmentation, significantly improving data processing efficiency. In experimental evaluations, MultiShotMaster demonstrated superior performance across multiple dimensions, including inter-shot consistency, narrative coherence, and reference image fidelity. Notably, the model won first place in the AAAI CVM Workshop competition, where its capabilities in world knowledge, camera motion, and cross-shot ID consistency were fully validated.
