DF-BSFNet: A bilateral synergistic fusion network with novel dynamic flow convolution for robust road extraction

Abstract

Accurate and robust road extraction with good continuity and completeness is crucial for the development of smart city and intelligent transportation. Remote sensing images and vehicle trajectories are attractive data sources with rich and complementary multimodal road information, and the fusion of them promises to significantly promote the performance of road extraction. However, existing studies on fusion-based road extraction suffer from the problems that the feature extraction modules pay little attention to the inherent morphology of roads, and the multimodal feature fusion techniques are too simple and superficial to fully and efficiently exploit the complementary information from different data sources, resulting in road predictions with poor continuity and limited performance. To this end, we propose a Bilateral Synergistic Fusion network with novel Dynamic Flow convolution, termed DF-BSFNet, which fully leverages the complementary road information from images and trajectories in a dual-mutual adaptive guidance and incremental refinement manner. First, we propose a novel Dynamic Flow Convolution (DFConv) that more adeptly and consciously captures the elongated and winding “flow” morphology of roads in complex scenarios, providing flexible and powerful capabilities for learning detail-heavy and robust road feature representations. Second, we develop two parallel modality-specific feature extractors with DFConv to extract hierarchical road features specific to images and trajectories, effectively exploiting the distinctive advantages of each modality. Third, we propose a Bilateral Synergistic Adaptive Feature Fusion (BSAFF) module which synthesizes the global-context and local-context of complementary multimodal road information and achieves a sophisticated feature fusion with dynamic guided-propagation and dual-mutual refinement. Extensive experiments on three road datasets demonstrate that our DF-BSFNet outperforms current state-of-the-art methods by a large margin in terms of continuity and accuracy.