Extracting data from maps: Lessons learned from the artificial intelligence for critical mineral assessment competition

Abstract

The U.S. Geological Survey (USGS), Defense Advanced Projects Research Agency (DARPA), Jet Propulsion Laboratory (JPL), and MITRE ran a 12-week machine learning competition aimed at accelerating development of AI tools for critical mineral assessments. The Artificial Intelligence for Critical Mineral Assessment Competition solicited innovative solutions for two challenges: 1) automated georeferencing of historical geologic and topographic maps, and 2) automated feature extraction from historical maps. Competitors used a new dataset of historical map images to train, validate, and evaluate their models. Automated georeferencing pipelines attained a median root-mean square error of 1.1 km. Prompt-based extraction (i.e., with user input) of polygons, polylines, and points from geologic maps yielded median F1 scores of 0.77, 0.56, 0.35, respectively. Geologic maps pose numerous challenges for AI workflows because they vary significantly. However, despite its short duration, the competition yielded promising results that have since spurred further innovation in this area and led to the development of new AI tools to semi-automate key, time-consuming parts of the assessment workflow.