A Cluster-Aggregate-Pool (CAP) ensemble algorithm for improved forecast performance of influenza-like illness

Seasonal influenza causes on average 425,000 hospitalizations and 32,000 deaths per year in the United States. Forecasts of influenza-like illness (ILI) — a surrogate for the proportion of patients infected with influenza — support public health decision making. The goal of an ensemble forecast of ILI is to increase accuracy and calibration compared to individual forecasts and to provide a single, cohesive prediction of future influenza. However, an ensemble may be composed of models that produce similar forecasts, causing issues with ensemble forecast performance and non-identifiability. To improve upon the above issues we propose a novel Cluster-Aggregate-Pool or ‘CAP’ ensemble algorithm that first groups together individual forecasts into clusters, aggregates forecasts that belong to the same cluster into a single forecast (called a cluster forecast), and then pools together cluster forecasts via a linear pool. We evaluated this algorithm on a benchmark dataset of 7 seasons of ILI plus forecasts generated by 27 individual models as part of the FluSight project. When compared to a non-CAP approach, we find that a CAP ensemble improves calibration by approximately 10% while maintaining similar accuracy to non-CAP alternatives. In addition, our CAP algorithm (i) generalizes past ensemble work associated with influenza forecasting and introduces a framework for future ensemble work, (ii) automatically accounts for missing forecasts from individual models, (iii) allows public health officials to participate in the ensemble by assigning individual models to clusters, and (iv) provide an additional signal about when peak influenza may be near.