StarVid: Enhancing Semantic Alignment in Video Diffusion Models via Spatial and SynTactic Guided Attention Refocusing

Abstract

Recent advances in text-to-video (T2V) generation with diffusion models have garnered significant attention. However, they typically perform well in scenes with a single object and motion, struggling in compositional scenarios with multiple objects and distinct motions to accurately reflect the semantic content of text prompts. To address these challenges, we propose StarVid, a plug-and-play, training-free method that improves semantic alignment between multiple subjects, their motions, and text prompts in T2V models. StarVid first employs large language models (LLMs) to perform two-stage motion trajectory planning based on the input prompt, providing object-level spatial priors that guide the generation process. These priors are then used to impose a spatial-aware loss that refines cross-attention (CA) maps, encouraging attention to be focused on distinct regions corresponding to different subjects. Furthermore, a syntax-guided contrastive constraint is introduced to explicitly enhance the association between verbs and their corresponding nouns in the CA maps, thereby improving motion-subject binding. Both qualitative and quantitative evaluations demonstrate that the proposed framework significantly outperforms baseline methods, delivering videos of higher quality with improved semantic consistency.