Abstract IA02: A brief overview of building and validating absolute risk models

Abstract

Statistical models that predict disease incidence, disease recurrence or mortality following disease onset have broad public health and clinical applications. Of great importance are models that predict absolute risk, namely the probability of a particular outcome, e.g. breast cancer, in the presence of competing causes of mortality. Although relative risks are useful for assessing the strength of risk factors, they are not nearly as useful as absolute risks for making clinical decisions or establishing policies for disease prevention. That is because such decisions or policies often weigh the favorable effects of an intervention on the disease of interest against the unfavorable effects that the intervention might have on other health outcomes. The common currency for such decisions is the (possibly weighted) absolute risk for each of the health outcomes in the presence and absence of intervention. First, I discuss various approaches to building absolute risk models from various data sources and illustrate them with absolute risk models for breast cancer and thyroid cancer. Before a risk prediction model can be recommended for clinical or public health applications, one needs to assess how good the predictions are. I will give an overview over various criteria for assessing the performance of a risk model. I assume that we have developed a risk model on training data and assess the performance of the model on independent test or validation data. This approach, termed external validation, provides a more rigorous assessment of the model than testing the model on the training data (internal validation); even though cross-validation techniques are available to reduce the over-optimism bias that can result from testing the model on the training data. I present general criteria for model assessment, such as calibration, predictive accuracy and classification accuracy, and discriminatory accuracy. Calibration measures how well the numbers of events predicted by a model agree with the observed events that arise in a cohort. Calibration is the most important general criterion, because if a model is not well calibrated, other criteria, such as discrimination, can be misleading. Discriminatory accuracy measures how well separated the distributions of risk are for cases and non-cases. Another approach is to tailor the criterion to the particular application. I will also present novel criteria for screening applications or high risk interventions. If losses can be specified in a well-defined decision problem, I will show how models can be assessed with respect to how much they reduce expected loss. Citation Format: Ruth Pfeiffer. A brief overview of building and validating absolute risk models. [abstract]. In: Proceedings of the AACR Special Conference: Improving Cancer Risk Prediction for Prevention and Early Detection; Nov 16-19, 2016; Orlando, FL. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2017;26(5 Suppl):Abstract nr IA02.