Modeling what matters: AI and the future of defense learning

Let’s be honest, artificial intelligence (AI) will change— or, rather, is already changing—so much. It would be easy, if uninspired, to fill this article with a laundry list. But rather than add to the existing litany of forecasts (many of which you can read in the chapters of this special edition), we’ll focus more narrowly. First, we’ve bound the question to learning in the defense domain, and second, we’ve challenged ourselves to target a single concept—to name the linchpin with greatest potential to have profound, paradigm-changing impacts. To give away the punchline, we’ve selected ‘‘the way we measure and evaluate.’’ Before we show our work, consider these definitions. Measure and evaluate refer to two sides of the same coin. Formally, measurement is the ‘‘quantitatively expressed reduction of uncertainty based on one or more observations’’ (p. 23). In other words, it refers to collected observations (no matter how fuzzy or incomplete) that help us fill-in (but not necessarily eliminate) uncertainty in a Claude Shannon ‘‘information theory’’ sort of way. Measurement goes hand-in-hand with evaluation. Evaluation is the process of interpreting the data collected from measurements, and for our purposes, we’ll say it covers all of the associated aggregation, transformation, analysis, and other activities needed to effectively use the measured data. Learning, as a formal concept, is related to—but notably distinct from—training and education. Those latter two terms, particularly in a defense context, are laden with connotations. ‘‘Training and education’’ refer to the organizational side of the experience, for instance, to the curriculum or the wargame delivered by a schoolhouse or training branch. They’re input-focused terms, and more than that, they tend to imply a formal learning context. In contrast, the term ‘‘learning’’ focuses on the individual (or team) side of the equation—the outcomes side. It describes any change in long-term memory that affects knowledge, skills, or behaviors, and it makes no distinction for the process through which it was acquired. 1. An operational perspective