Lightweight and efficient feature fusion real-time semantic segmentation network

The increasing demand for real-time performance in semantic segmentation for the field of autonomous driving has prompted a significant focus on the trade-off between speed and accuracy. Recently, many real-time semantic segmentation networks have opted for lightweight classification networks as their backbone. However, their lack of specificity for real-time semantic segmentation tasks compromises their ability to extract advanced semantic information effectively. This paper introduces the LAFFNet, a lightweight and efficient feature-fusion real-time semantic segmentation network. We devised a novel lightweight feature extraction block (LEB) to construct the encoder part, employing a combination of deep convolution and dilated convolution to extract local and global semantic features with minimal parameters, thereby enhancing feature map characterization. Additionally, we propose a coarse feature extractor block (CFEB) to recover lost local details during encoding and improve connectivity between encoding and decoding parts. In the decoding phase, we introduce the bilateral feature fusion block (BFFB), leveraging features from different inference stages to enhance the model’s ability to capture multi-scale features and conduct efficient feature fusion operations. Without pre-training, LAFFNet achieves a processing speed of 63.7 FPS on high-resolution (1024 × 2048) images from the Cityscapes dataset, with an mIoU of 77.06%. On the Camvid dataset, the model performs equally well, reaching 107.4 FPS with an mIoU of 68.29%. Notably, the model contains only 0.96 million parameters, demonstrating its exceptional efficiency in lightweight network design. These results demonstrate that LAFFNet achieves an optimal balance between accuracy and speed, providing an effective and precise solution for real-time semantic segmentation tasks.