Evaluating space-based methane detection through a one-year blind test

Abstract

Space-based remote sensing allows detection and monitoring of large, concentrated methane sources worldwide1. However, independent evaluation of instruments and algorithms is limited, and new satellites and artificial intelligence techniques have not been tested under controlled, single-blind conditions2. Here we conduct a year-long, single-blind test involving 746 events: 481 methane releases of 9.5–1,183.7 kg h−1 and 265 blank controls. We released metered methane quantities in Arizona, USA, coinciding with overpasses from 12 government, commercial and non-profit satellites. Fifteen analysis teams from nine countries generated more than 2,250 estimates, making this, to our knowledge, the largest controlled test so far and one of the first evaluations of the instruments EMIT, Tanager-1 (Carbon Mapper) and GHOSt. Performance is heterogeneous: ‘built-for-purpose’ instruments designed to target methane demonstrate excellent detection performance (92–100% true positive rate, 0–11% false positive rate) and quantification accuracy (slope 0.83–1.00, 44% mean absolute percentage error). Instruments repurposed for methane sensing perform more variably (detection rates 62–88%, 46% mean absolute percentage error); still, many demonstrate significant skill. Broad land scanners detect poorly at tested rates but have consistent global coverage and historical archives, motivating additional work. Artificial intelligence-based methods show promise, but caution is warranted; 79% of false positives emerge from these methods. Ground-measured cloud cover >20% degraded performance across systems, raising concerns for use in some locations. Our findings show that satellite-based measurements can achieve accurate detection and quantification of methane point sources when the source location is approximately known. A year-long single-blind study shows that multiple satellites can accurately detect and quantify methane point-source emissions.