MagNet—A Data‐Science Competition to Predict Disturbance Storm‐Time Index (Dst) From Solar Wind Data

Abstract

Abstract Enhanced interaction between solar‐wind and Earth's magnetosphere can cause space weather and geomagnetic storms that have the potential to damage critical technologies, such as magnetic navigation, radio communications, and power grids. The severity of a geomagnetic storm is measured using the disturbance‐storm‐time ( Dst ) index. The Dst index is calculated by averaging the horizontal component of the magnetic field observed at four near‐equatorial observatories and is used to drive geomagnetic disturbance models. As a key specification of the magnetospheric dynamics, the Dst index is used to drive geomagnetic disturbance models such as the High Definition Geomagnetic Model—Real Time. Since 1975, forecasting models have been proposed to forecast Dst solely from solar wind observations at the Lagrangian‐1 position. However, while the recent Machine‐Learning (ML) models generally perform better than other approaches, many are unsuitable for operational use. Recent exponential growth in data‐science research and the democratization of ML tools have opened up the possibility of crowd‐sourcing specific problem‐solving tasks with clear constraints and evaluation metrics. To this end, National Oceanic and Atmospheric Administration (NOAA)'s National Centers for Environmental Information and the University of Colorado's Cooperative Institute for Research in Environmental Sciences conducted an open data‐science challenge called “MagNet: Model the Geomagnetic Field.” The challenge attracted 622 participants, resulting in 1,197 model submissions that used various ML approaches. The top models that met the evaluation criteria are operationally viable and retrainable and suitable for NOAA's operational needs. The paper summarizes the competition results and lessons learned.