A dual of Transformer features-related map-intelligent generation method

Abstract

目的 现有的地图智能生成技术没有考虑到地图生成任务存在的地理要素类内差异性和地理要素域间差异性，这使得生成的地图质量难以满足实际需要。针对地理要素类内差异性和地理要素域间差异性，提出了一种Transformer特征引导的双阶段地图智能生成方法。方法 首先基于最新的Transformer网络，设计了一个基于该网络的特征提取模块，该模块提取遥感图像中的地理要素特征用于引导地图生成，解决了地理要素类内差异性导致的地图生成困难的问题。然后设计双阶段生成框架，该框架具备两个生成对抗网络，第1个生成对抗网络为初步生成对抗网络，利用遥感图像和Transformer特征得到初步的地图图像；第2个生成对抗网络为精修生成对抗网络利用初步地图图像生成高质量的精修地图图像，缓解了地理要素域间差异性导致的地图地理要素生成不准确问题。结果 在AIDOMG（aerial image dataset for online map generation）数据集上的9个区域进行了实验，与10种经典的和最新方法进行了比较，提出方法取得了最优的结果。其中，在海口区域，相比于Creative GAN方法，FID （Frechet inception distance）值降低了16.0%，WD （Wasserstein distance）降低了4.2%，1-NN （1-nearest neighbor）降低了5.9%；在巴黎区域，相比于Creative GAN方法，FID值降低了2.9%，WD降低了1.0%，1-NN降低了2.1%。结论 提出的Transformer特征引导的双阶段地图智能生成方法通过高质量的Transformer特征引导和双阶段生成框架解决了地理要素类内差异性和地理要素域间差异性所带来的地图生成质量较差的问题。;Objective Map intelligent generation technique is focused on generating map images quickly and cost efficiently. For existing intelligent map generation technique,to get quick-responsed and low-cost map generation,remote sensing image is taken as the input,and its generative adversarial network(GAN) is used to generate the corresponding map image. Inevitably,it is challenged that the intra-class differences within geographical elements in remote sensing images and the differences of geographical elements between domains in the map generation task are still not involved in. The intra-class difference of geographical elements refers that similar geographical elements in remote sensing images have several of appearances,which are difficult to be interpreted. Geographical elements segmentation is required for map generation in relevance to melting obvious intra-class differences into corresponding categories. The difference of geographical elements between different domains means that the corresponding geographical elements in remote sensing images and map images are not exactly matched well. For example,the edges of vegetation elements in remote sensing images are irregular, while the edges of vegetation elements in map images are flat. Another challenge for map generation is to generate and keep consistency to the features of map elements. Aiming at the intra-class difference of geographical elements and the superposition of geographical elements,we develop a dual of map-intelligent generation method based on Transformer features. Method The model consists of three sorts of modules relevant to feature extraction,preliminary and refined generative adversarial contexts. First,feature extraction module is developed based on the latest Transformer network. It consists of a backbone and segmentation branch in terms of Swin-Transformer structure. Self-attention mechanism based Transformer can be used to construct the global relationship of the image,and it has a larger receptive field and it can extract feature information effectively. The segmentation branch is composed of a pyramid pooling module(PPM) and a feature pyramid network(FPN). To get more effective geographic element features,feature pyramid is employed to extract multi-level feature information,and the high-level geographic element semantic information can be integrated into the middle-level and low-level geographic element semantic information,and the PPM is used to introduce the global semantic information as well. Next,feature information is sent to the segmentation branch,which uses the actual segmentation results as a guidance to generate effective geographical element features. To guide map generation and resolve the problem of map generation caused by the differences in geographical elements,this module can be used to extract the features of geographical elements in remote sensing images. Third,the preliminary generative adversarial module has a preliminary generator and a discriminator. The preliminary generator is a multi-scale generator,consisting of a local generator and a global generator, and it is used to generate the high-resolution images. Both of local and global generators are linked to encoder/decoder structures. The input of the preliminary generator is derived of remote sensing image and geographical element features, and the output is originated from preliminary map image. The discriminator is also recognized as a multi-scale discriminator,which consists of three sorts of sub discriminators for the high-resolution images. The input of the discriminator is the generated map and the real map,and the output is the single channel confidence map. Finally,a refined generator is used for refined generative adversarial module,and a discriminator with the preliminary generative adversarial module is shared in as well. The structure of the refined generator is same as the preliminary generator,which is also as a multi-scale generator in terms of local and global generators. The input of the refinement generator is originated from a preliminary map image and the output is derived of a fine map image. A dual of generation framework is constructed in terms of refined and preliminary generative adversarial-related modules. In general,to obtain preliminary map images,the preliminary generative adversarial module is as inputs based on remote sensing images and geographical element features. The preliminary map image is rough,and there are incomplete geographical elements,such as uneven road edges and fractures. For the refined generative adversarial module,to learn the geometric characteristics of geographical elements in the real map,obtain highquality fine map images,and alleviate the problem of inaccurate local map generation caused by the differences of geographical elements between domains,the generated primary map image is taken as the input,and the real map is taken as the guide as well. Result Experiments are carried out on 9 regions on the aerial image dataset for online map generation (AIDOMG) dataset in comparison with 10 sort of popular methods. For the Haikou area,Frechet inception distance(FID) is reduced by 16. 0%,Wasserstein distance(WD) is reduced by 4. 2%,and the 1-nearest neighbor(1-NN) is reduced by 5. 9% as well. For the Paris area,FID is decreased by 2. 9%,WD is decreased by 1. 0%,and 1-NN decreased by 2. 1% simultaneously. Comparative analyses demonstrate that our method proposed can improve the results of map generation effectively. At the same time,ablation studies of the model can show the effectiveness of each module,and each module can be added and the model results is improved gradually as well. Conclusion To solve the problem of poor map generation quality caused by the intra-class inconsistency of geographical elements effectively,a dual of Transformer features-related map-intelligent generation method is proposed,and the differences of geographical elements between domains can be illustrated via high-quality Transformer-guided feature and a dual of generation framework further.