Jia-Xing Zhong, Kai Lu, Jiaojiao Ye, Niki Trigoni, Andrew Markham
{"title":"MotionStruct4D: Discovering Motion Structure of Gaussian Splatting for Video-to-4D Generation","authors":"Jia-Xing Zhong, Kai Lu, Jiaojiao Ye, Niki Trigoni, Andrew Markham","doi":"10.1007/s11263-026-02992-z","DOIUrl":"https://doi.org/10.1007/s11263-026-02992-z","url":null,"abstract":"Monocular Video-to-4D generation faces the fundamental challenge of inferring plausible 3D geometry and motion from limited single-view inputs. We present MotionStruct4D, a novel approach that discovers and exploits the underlying motion structure of 3D Gaussian Splatting for high-quality video-to-4D generation. Our key insight is that real-world motion can be effectively decomposed into coarse rigid transformations that capture principal movements, complemented by detailed non-rigid deformations that account for fine-grained details. MotionStruct4D introduces: (1) a self-supervised motion structure discovery module that identifies quasi-rigid parts by preserving spatiotemporal relationships without explicit 3D supervision, and (2) a weighted dense-to-sparse optimization architecture that transitions from dense per-Gaussian deformation to sparse control points, effectively integrating rigid and non-rigid motion components through adaptive weighted fusion. This design addresses the parametric imbalance between rigid and non-rigid motions and effectively models 3D movements across different kinematic patterns. To evaluate our approach on challenging scenarios, we curate a comprehensive benchmark dataset featuring substantial object displacement and diverse articulated motion patterns. Experimental results demonstrate MotionStruct4D’s superior performance in motion fidelity and novel viewpoint synthesis quality, while also providing interpretable motion structure decomposition that reveals meaningful quasi-rigid part segmentation.","PeriodicalId":13752,"journal":{"name":"International Journal of Computer Vision","volume":"43 1","pages":""},"PeriodicalIF":19.5,"publicationDate":"2026-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148884075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ShapeRhythm: Personalized and Harmonic Dance Generation Bridging Diverse Bodies and Melodies","authors":"Muyi Sun,Zezhao Tian,Xingqun Qi,Chen Su,Xurui Hu,Qifeng Liu,Man Zhang,Zhenan Sun,Caifeng Shan","doi":"10.1007/s11263-026-03017-5","DOIUrl":"https://doi.org/10.1007/s11263-026-03017-5","url":null,"abstract":"Abstract Deriving vivid and expressive 3D human dance from music signals has gained tremendous progress in virtual avatar animation. Existing approaches typically overlook the diversity of human body shapes and generate dance motions using standardized body representations, without explicitly modeling the intrinsic relationship between body shape variations (i.e., thinness or plumpness) and musical rhythm. As a result, the synthesized dance motion lacks both personalization and harmony. Moreover, the absence of high-quality human dance datasets, particularly those incorporating bodyshape variations and annotations, also poses a significant challenge for this problem. To address this issue, we first newly compile a large-scale 3D human dance dataset, named ShapeDancer , which encompasses 13,574 s of data paired with detailed body shape variations. Additionally, we propose ShapeRhythm , a novel framework that allows personalized and harmonic dance generation, bridging diverse human bodies with expressive melodies. Our ShapeRhythm framework is built upon unifying the music signals and shape-aware text descriptions as conditions. Specifically, to enhance the dance personalization, we propose FreeShape , a delicate shape similarity matching approach which realizes freestyle and open-vocabulary text-guided bodyshape conversion. Furthermore, we design FreeDance to diversify and harmonize the dance motion generation, while preserving the shape-aware temporal consistency with music rhythm. Meanwhile, a fine-grained physics-guided Foot-Ground Interaction Constraint (FGI) is introduced to alleviate foot-skating artifacts and enhance detailed foot rhythm coordination. Extensive experiments conducted on our newly collected ShapeDancer dataset demonstrate that ShapeRhythm achieves state-of-the-art performance, realizing personalized and harmonic dance generation. More details are available on our project page .","PeriodicalId":13752,"journal":{"name":"International Journal of Computer Vision","volume":"31 14 1","pages":""},"PeriodicalIF":19.5,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148932874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}