LLM-guided fuzzy kinematic modeling for resolving kinematic uncertainties and linguistic ambiguities in text-to-motion generation

Abstract

Generating realistic and coherent human motions from text descriptions is essential for applications in computer vision, computer animations, and digital environments. However, existing text-to-motion generation models often overlook kinematic uncertainties and linguistic ambiguities, leading to unnatural and misaligned motion sequences. To address these issues, we propose a novel framework that integrates fuzzy kinematic modeling with large language model (LLM) guidance to jointly model kinematic uncertainties and resolve linguistic ambiguities. Our approach first extracts rich kinematic attributes from raw motion data and converts them into fuzzy kinematic facts (FKFs), which serve as an uncertainty-aware motion representation across different kinematic hierarchies. Simultaneously, we refine ambiguous text descriptions by extracting contextual terms using LLM-guided few-shot in-context learning, enhancing text with additional semantic clarity. These FKFs and contextual terms are then used to train a diffusion-based motion generation model, ensuring semantically accurate and physically plausible motion synthesis. To further enhance kinematic structural consistency in FKF representations, we introduce a Graph-Augmented Self-Attention (GASA) module, which injects spatio-temporal relational constraints into the diffusion process, improving motion coherence and structural integrity. Evaluations on HumanML3D and KIT-ML datasets demonstrate that our method outperforms state-of-the-art models, achieving the lowest FID scores (0.052 and 0.091) and reducing kinematic uncertainty footprint by 21.1% and 17.7%, respectively. The source code and additional resources are publicly available at https://alimanjotho.github.io/llm-fqk-t2m.