DBMSTN: A Dual Branch Multiscale Spatio-Temporal Network for dim-small target detection in infrared image

Abstract

Addressing the challenging task of infrared dim and small target (IDST) detection in complex background, which is a major topic in infrared image processing, we propose a Dual Branch Multiscale Spatio-Temporal Network (DBMSTN) to suppress complex background and effectively extract targets’ geometric and motion features. Firstly, DBMSTN utilizes a multiscale spatial feature extraction module that extracts inter-frame difference and saliency feature to highlight small targets at different scales and suppress complex backgrounds. Secondly, the DBMSTN contains a dual-branch spatio-temporal feature extraction module which is designed with improved gating unit in convolutional LSTM (ConvLSTM) to enhance the extraction of motion features to cope with their uncertainty. In addition, DBMSTN achieves a better performance using a fusing module that fuses multilevel spatio-temporal features. It also employs the weighted mean squared error (MSE) loss function with adjustable weights of positive and negative samples to solve the data imbalance problem. Experiments based on two public benchmarks verify that DBMSTN outperforms the state-of-the-art metrics and achieves the highest F1 up to 0.9860, also effectively extracts spatio-temporal features of targets with different speeds.