SMOL-MapSeg: Segmenting historical maps by legend-oriented prompting

Abstract

Historical maps offer valuable insights into Earth’s surface changes but pose challenges for modern segmentation models due to inconsistent visual styles and symbols. While deep learning models like UNet and pre-trained foundation models perform well in domains such as autonomous driving and medical imaging, they struggle with the variability of historical maps, where similar concepts appear in diverse forms. To overcome this, we propose legend-oriented prompting, which provides an explicit image–label pair to guide models in identifying the target visual patterns shown in the prompt. This allows users to define and segment target patterns on demand, enabling flexible legend-based segmentation that mimics human map reading. Our approach replaces the prompt encoder of the Segment Anything Model (SAM) with the newly proposed prompting mechanism and fine-tunes it on historical maps, creating the SMOL-MapSeg (Show Me One Label¹) model. Unlike existing SAM-based fine-tuning methods that are class-agnostic or limited to fixed classes, SMOL-MapSeg is able to segment any target class across arbitrary datasets according to the class patterns defined by the prompt. Experiments show that a single SMOL-MapSeg model can accurately segment different classes, significantly outperforming baseline models. Moreover, it demonstrates strong generalization, even with limited training data, highlighting its potential for scalable and adaptable historical map analysis.