Validity and Validation of Computer Simulations—A Methodological Inquiry with Application to Integrated Assessment Models

Abstract

Our purpose is to advance a reasoned perspective on the scientific validity of computer simulation, using an example—integrated assessment modeling of climate change and its projected impacts—that is itself of great and urgent interest to policy in the real world. The spirited and continuing debate on the scientific status of integrated assessment models (IAMs) of global climate change has been conducted mostly among climate change modelers and users seeking guidance for climate policy. However, it raises a number and variety of issues that have been addressed, with various degrees of success, in other literature. The literature on methodology of simulation was mostly skeptical at the outset but has become more nuanced, casting light on some key issues relating to the validity and evidentiary standing of climate change IAMs (CC-IAMs). We argue that the goal of validation is credence, i.e., confidence or justified belief in model projections, and that validation is a matter of degree: (perfect) validity is best viewed as aspirational and, other things equal, it makes sense to seek more rather than less validation. We offer several conclusions. The literature on computer simulation has become less skeptical and more inclined to recognize that simulations are capable of providing evidence, albeit a different kind of evidence than, say, observation and experiments. CC-IAMs model an enormously complex system of systems and must respond to several challenges that include building more transparent models and addressing deep uncertainty credibly. Drawing on the contributions of philosophers of science and introspective practitioners, we offer guidance for enhancing the credibility of CC-IAMs and computer simulation more generally.