High-performance fire detection framework based on feature enhancement and multimodal fusion

Abstract

Fire detection technology has become increasingly critical in the context of rising global fire threats and extreme weather conditions. Traditional methods rely on single-modal sensors and conventional image processing techniques and often struggle with complex environmental variations and background interference. This study proposes an innovative multimodal fire detection framework that integrates advanced deep learning techniques to address these limitations. By leveraging a comprehensive approach that combines YOLOv8-based object detection, HSV color space enhancement, completed local binary pattern (CLBP) texture analysis, and a novel dynamic feature enhancement module (DFEM), the proposed method significantly improves fire detection accuracy and robustness. This research introduces a sophisticated multimodal fusion strategy that systematically processes fire-related features across multiple domains. A key innovation is the cross-modality fusion Mamba (CMFM) module, which employs efficient channel attention (ECA) and an efficient 2D-selective scan module (E2DSM) to dynamically integrate and refine features from different modalities. Experimental validation was conducted on a dataset that we collected, which was supplemented by data collected via real-world robotic image acquisition in diverse environments, including forests, corridors, and outdoor settings. The proposed method demonstrated exceptional performance, with a precision of 96.4%, a recall of 95.7%, and an overall accuracy of 95.8%, outperforming state-of-the-art models such as VGG16, ResNet50, YOLOv5, and YOLOv8. Ablation studies further validated the contribution of each module and highlighted the framework’s robust feature enhancement and fusion capabilities.