Evaluation of a decision support system using Bayesian network modeling in an applied Multi-INT surveillance environment.

Abstract

Sensemaking and decision-making are fundamental components of applied Intelligence, Surveillance, and Reconnaissance (ISR). Analysts acquire information from multiple sources over a period of hours, days, or even over the scale of months or years, that must be interpreted and integrated to predict future adversarial events. Sensemaking is essential for developing an appropriate mental model that will lead to accurate predictions sooner. Decision Support Systems (DSS) are one proposed solution to improve analyst decision-making outcomes by leveraging computers to conduct calculations that may be difficult for human operators and provide recommendations. In this study, we tested two simulated DSS that were informed by a Bayesian Network Model as a potential prediction-assistive tool. Participants completed a simulated multi-day, multi-source intelligence task and were asked to make predictions regarding five potential outcomes on each day. Participants in both DSS conditions were able to converge on the correct solution significantly faster than the control group, and between 36-44% more of the sample was able to reach the correct conclusion. Furthermore, we found that a DSS representing projected outcome probabilities as numerical, rather than using verbal ordinal labels, were better able to differentiate which outcomes were extremely unlikely than the control group or verbal-probability DSS.