Improving External Validity of Machine Learning, Reduced Form, and Structural Macroeconomic Models using Panel Data

Abstract

We show that adding countries as a panel dimension to macroeconomic data can statistically significantly improve the generalization ability of structural and reduced-form models, as well as allow machine learning methods to outperform these and other macroeconomic forecasting models. Using GDP forecasts for evaluation, this procedure reduces root mean squared error (RMSE) by 12% across horizons and models for certain reduced-form models and by 24% across horizons for structural DSGE models. Removing US data from the training set and forecasting out-of-sample country-wise, we show that both reduced form and structural models become more policy invariant, and outperform a baseline model that uses US data only. Finally, given the comparative advantage of "nonparametric" machine learning forecasting models in a data-rich regime, we demonstrate that our recurrent neural network (RNN) model and automated machine learning (AutoML) approach outperforms all baseline economic models in this regime. Robustness checks indicate that machine learning outperformance is reproducible, numerically stable, and generalizes across models.