The Q $$ Q $$ -Score: A Magnitude-Weighted Goodness-of-Fit Score for Earthquake Forecasting

Abstract

Accurate forecasting of large earthquakes is of great importance, yet most current earthquake forecast evaluation metrics, such as the log-likelihood score, do not give additional weight to large-magnitude events. To this end, magnitude-weighted goodness-of-fit scores for earthquake forecasting have recently been introduced, such as potency-weighted log-likelihood and the $Q$-score. In this article, we investigate properties of the $Q$-score, which is a quotient emphasizing model fit for the largest 5% of earthquakes. We explore the theoretical properties of the $Q$-score, demonstrating that under certain null conditions, the expectations of the numerator and denominator are equal and thus the expectation of $Q$ is 1 in a ratio sense. Additionally, the score satisfies a law of large numbers. We evaluated the $Q$-score of 21 next-day gridded earthquake forecasts for California, provided by the Collaborative for the Study of Earthquake Predictability (CSEP) for the years 2012, 2014, and 2017. We also calculate the log-likelihood scores of the forecasts to obtain a more comprehensive evaluation of how well different models perform, both for predicting the largest events and also in terms of overall model fit.