Multimodal urban areas of interest generation via remote sensing imagery and geographical prior

Urban area-of-interest (AOI) refers to an integrated urban functional zone with defined polygonal boundaries. The rapid development of urban commerce has led to increasing demands for highly accurate and timely AOI data. However, existing research primarily focuses on coarse-grained functional zones for urban planning or regional economic analysis, and often neglects AOI’s expiration in the real world. They fail to fulfill the precision requirements of Mobile Internet Online-to-Offline (O2O) businesses. These businesses require AOI boundary accuracy down to a specific community, school, or hospital. In this paper, we propose a fully end-to-end multimodal AOI TRansformer (AOITR) model designed for simultaneously detecting accurate AOI boundaries and validating AOI’s reliability by leveraging remote sensing imagery coupled with geographical prior. Unlike conventional AOI generation methods, such as the Road-cut method that segments road networks at various levels, our approach diverges from semantic segmentation algorithms that depend on pixel-level classification. Instead, our AOITR begins by selecting a point-of-interest (POI) of specific category, which can be easily obtained via web crawler, and uses it to retrieve corresponding remote sensing imagery and geographical prior such as entrance POIs and road nodes. This information helps to build a multimodal detection model based on transformer encoder-decoder architecture to regress the accurate AOI polygon. Additionally, we utilize the dynamic features from human mobility, nearby POIs, and logistics addresses for AOI reliability evaluation via a cascaded network module. The experimental results reveal that our algorithm achieves a significant improvement on Intersection over Union (IoU) metric, surpassing previous methods by a large margin. Furthermore, the AOIs produced by AOITR have substantially enriched our AOI library and have been successfully applied on over 10 different O2O scenarios including Alipay’s face scan payment service.