Fall-Mamba: A Multimodal Fusion and Masked Mamba-Based Approach for Fall Detection

Falls are a leading cause of injury and death among the elderly, making fall detection critically important. Traditional wearable sensors and environmental devices have limitations in terms of comfort, convenience, and accuracy. With the advancement of artificial intelligence and the Internet of Things (IoT), camera-based fall detection has become a research focus, but challenges, such as occlusion and poor lighting conditions remain. To address these issues, this study introduces an innovative model named Fall-Mamba. Compared to previous approaches, Fall-Mamba utilizes a Cross-Attention mechanism to fuse video and audio data, significantly enhancing the comprehensive understanding and detection performance of fall events. Additionally, the model incorporates a multihead temporal attention mechanism and the Frame Masking strategy, improving its ability to capture key frames and increasing its robustness. Extensive experiments conducted on multiview, multiscene datasets, including Le2i fall detection dataset, URFD, and Multicam, demonstrate the superior performance of Fall-Mamba, achieving an accuracy of 99.63% and exhibiting high robustness. This technology provides strong protection for the safety of the elderly in IoT-enabled smart homes. The code has been published at https://github.com/DHUspeech/fall-mamba.