A Framework for Evaluating the Use of Surveillance Systems for Short-Term Influenza Forecasting

Abstract

Background

Public health surveillance systems need to monitor influenza activity and guide measures to mitigate its high impact on morbidity, mortality and healthcare systems. There is an increasing expectation that surveillance data will support the modeling of future short-term disease scenarios using artificial intelligence (AI) and machine learning (ML). This study examines how influenza surveillance can support AI/ML-based short-term forecasting for influenza at the community and hospital levels in a high-income country setting (Aotearoa/New Zealand).

Methods

This study used a two-phase approach. The first phase involved a comprehensive review of government reports, official websites, and literature to characterize existing influenza surveillance systems. The second phase evaluated systems against eight key attributes—timeliness, sensitivity, specificity, representativeness, coverage, robustness, completeness, and historical data—using a five-level ranking system. Attribute selection was informed by experts' knowledge, ML requirements, and established frameworks. Weighted scores for training and short-term forecasting capabilities were calculated to determine alignment with AI/ML requirements.

Results

The Southern Hemisphere Influenza and Vaccine Effectiveness Research and Surveillance (SHIVERS) community cohort and Severe Acute Respiratory Infection (SARI) hospital surveillance emerged as the most useful systems, achieving the highest scores in both training and short-term forecasting in community and hospital settings, respectively. The National Minimum Dataset of hospitalizations and mortality datasets demonstrated strong training potential but are limited in short-term forecasting due to timeliness constraints. Additionally, laboratory-based surveillance performs a useful role in bridging community and hospital datasets.

Conclusions

A set of key attributes is useful for assessing which influenza surveillance systems are best aligned with AI/ML training and short-term forecasting requirements. These attributes distinguished systems that are likely to be the most suitable for modeling future short-term disease scenarios for influenza at the community and hospital levels in New Zealand. Integrating these data sources could enhance influenza forecasts to improve public health responses and intervention planning.