A New Paradigm for Video Interaction: Introducing the REVEL Task
With autoregressive Video Diffusion Models (VDMs) becoming mainstream, the fluidity and realism of AI-generated videos have seen significant progress. However, the user’s need for real-time, fine-grained control during the generation process has remained unfulfilled. Current technologies are often plagued by bottlenecks such as limited functionality, inability to adapt to streaming generation, and an imbalance between cost and quality. This forces users to interrupt and regenerate videos upon discovering artifacts, severely limiting creative efficiency and freedom.
To address this industry pain point, a research team from Nanyang Technological University, Singapore, and Hefei University of Technology, presented at ICLR 2026, has defined a new task called “stReaming drag-oriEnted interactiVe vidEo manipuLation” (REVEL). The core goal of REVEL is to empower users to apply drag commands to any frame at any point during autoregressive video generation, ensuring that the subsequently generated video content maintains temporal consistency and visual naturalness with the modification.
This task paradigm is the first to unify drag-based video manipulation into two core types:
- Editing: Directly modifying objects or scenes in already generated video frames.
- Animation: Generating subsequent dynamic video clips based on the current frame, following the user’s drag trajectory.
The establishment of the REVEL task provides a unified technical standard and evaluation benchmark for the long-fragmented field of interactive video editing. It aims to break the limitations of traditional methods, where editing and animation functions are separate and operation types are restricted.
DragStream: Solving Two Core Technical Challenges in Streaming Drag Editing
To achieve the goals of the REVEL task, the research team developed DragStream, a training-free method. This method directly confronts the two fundamental challenges of introducing drag operations into streaming video generation: Latent Distribution Shift and Contextual Interference.
Latent Distribution Shift: In autoregressive models, drag-based modifications to a single frame introduce a mathematical “perturbation.” This perturbation is progressively amplified during the generation of subsequent frames, causing the video’s latent space encoding to drift away from the effective data distribution learned during model training. The end result is unexpected termination of the drag process or even severe distortions, such as incorrect object colors and shapes.
Contextual Interference: Streaming generation models heavily rely on preceding frames as contextual information to predict the next frame. When a user drags an object, the model still “sees” the visual information of the object at its original position when referencing previous frames. This spatio-temporal information conflict can easily mislead the model, leading to the generation of duplicated object parts or obvious visual artifacts near the dragged area.
DragStream effectively solves these problems without retraining the model, through two key technical innovations.
Innovation 1: Adaptive Distribution Self-Correction (ADSR)
To counter latent distribution shift, DragStream introduces the ADSR strategy. Its core idea is to use the temporal continuity between video frames to constrain the statistical properties of the latent codes. After each iteration of drag optimization on the current frame, this strategy obtains the mean and standard deviation of the latent embeddings from its preceding adjacent frames. It then uses this relatively stable statistical data to correct the perturbed latent code distribution of the current frame. This “self-correction” mechanism acts like a tether, continuously pulling the deviated latent codes back into the normal distribution range, thus ensuring the stability of the drag process and the consistency of the edited object’s attributes.
Innovation 2: Spatio-Frequency Selective Optimization (SFSO)
To leverage contextual information while suppressing the interference it causes, the researchers designed the SFSO mechanism, which performs selective optimization from both spatial and frequency dimensions.
Switchable Frequency Selection (SFS): This strategy tackles the problem from the frequency domain to balance details and artifacts. Within the self-attention blocks of a DiT (Diffusion Transformer) model, it transforms features into the frequency domain via a Fourier Transform. It then uses a Butterworth filter with randomly selected cutoff frequencies to filter the features. This allows the optimization process to dynamically weigh high-frequency details (which can introduce noise) against low-frequency structures (which are stable but lack detail), preserving fine visual features while suppressing the generation of artifacts at the source.
Critically-driven Spatial Selection (CSS): This strategy constrains the optimization scope in the spatial domain. It generates a Gaussian map centered on the drag points to attenuate the weights for gradient backpropagation. This means the optimization process is highly focused on the user-specified drag regions, preventing gradient “leakage” into the background and other non-target areas. This effectively prevents unnecessary changes to the rest of the frame and ensures editing precision.
Comprehensive Experimental Validation: SOTA Performance and Generalization
To comprehensively evaluate DragStream’s performance, the research team constructed a benchmark dataset of 204 videos, covering diverse scenes and drag trajectories. The experiments compared DragStream with SOTA methods (like DragVideo and SG-I2V) adapted for streaming generation.
On quantitative metrics, the study used Object Motion Consistency (ObjMC), Drag Accuracy and Impact (DAI), Fréchet Video Distance (FVD), and Fréchet Inception Distance (FID) for evaluation. The results showed that DragStream significantly outperformed the comparison methods on all four metrics, proving that its generated videos achieve state-of-the-art levels in drag accuracy, image quality, and overall fluidity.
In visual comparisons, DragStream can accurately realize the user’s intent under various operations such as translation, deformation, and 2D/3D rotation. The generated frames are natural, with almost no artifacts or structural distortions. In contrast, other methods commonly suffer from drag failures, object shape distortions, or severe artifacts.
Furthermore, DragStream demonstrates strong generalization ability. It can successfully handle complex scenarios like objects re-emerging after occlusion and re-entering the frame after moving out. In long video generation up to 20 seconds, it maintains stable output quality even after multiple consecutive drags. Its model-agnostic design allows it to be integrated in a plug-and-play manner into different autoregressive VDM backbones like CausVid, offering high flexibility and practical value.
Conclusion and Future Outlook
DragStream is not just a high-performance streaming video editing tool; more importantly, by defining the REVEL task, it establishes a clear and complete technical paradigm for the field of interactive AI video generation. Its emergence is poised to fundamentally change the inefficient “generate-reject-regenerate” cycle of AIGC video creation, transforming creators from passive “selectors” into active “controllers.”
The study’s training-free, plug-and-play nature significantly lowers the barrier to entry for advanced video editing technology, paving the way for the development of consumer-grade, real-time interactive video generation tools. Although the method still has limitations in handling extreme drag commands that violate physical common sense, it undoubtedly opens a new chapter for freedom and interactivity in AIGC video creation, heralding a new “What You See Is What You Get” (WYSIWYG) era of video generation.
