A Verification Approach Used in Developing the Rapid Refresh and Other Numerical Weather Prediction Models

Developing and improving numerical weather prediction models such as the Rapid Refresh (RAP) and High-Resolution Rapid Refresh (HRRR) requires a well-designed, easy-to-use evaluation capability using observations. Owing to the very complex nonlinear interactions between the data assimilation system and the representation of various physics components in the model, changes to one aspect of the modeling system to address a particular shortcoming within the model may have detrimental impacts in another area. Thus, the model verification approach used in the Global Systems Division of the NOAA Earth System Research Laboratory—which actively develops the RAP and HRRR models and other forecasting systems—is designed to allow hypothesis-driven testing of different aspects of the model using observations. In this approach, model changes easily and quickly can be quantified by automatically comparing simulated geophysical variables against many different types of observations that are collected operationally by various agencies, including the National Weather Service. We have implemented this approach in the Model Analysis Tool Suite (MATS). A key aspect of MATS is the use of a database-driven system that stores partial sums of model minus observation pairs over specified geographical regions in order to reduce the dimensionality of the data and, thus, improve the response time of the system. These partial sums are created and stored in a manner that allows the data to be visualized in different ways, thereby providing new insights into the ability of that particular version of the model to replicate the observed atmospheric conditions.