PADTHAI-MM: Principles-based approach for designing trustworthy, human-centered AI using the MAST methodology

Despite an extensive body of literature on trust in technology, designing trustworthy AI systems for high-stakes decision domains remains a significant challenge. Widely used system design guidelines and tools are rarely attuned to domain-specific trustworthiness principles. In this study, we introduce a design framework to address this gap within intelligence analytic tasks, called the Principles-based Approach for Designing Trustworthy, Human-centered AI using the MAST Methodology (PADTHAI-MM). PADTHAI-MM builds on the Multisource AI Scorecard Table (MAST), an AI decision support system evaluation tool designed in accordance to the U.S. Intelligence Community's standards for system trustworthiness. We demonstrate PADTHAI-MM in our development of the Reporting Assistant for Defense and Intelligence Tasks (READIT), a research platform that leverages data visualizations and natural language processing-based text analysis to emulate AI-enabled intelligence reporting aids. To empirically assess the efficacy of PADTHAI-MM, we developed two versions of READIT for comparison: a “High-MAST” version, which incorporates AI contextual information and explanations, and a “Low-MAST” version, designed to be akin to inscrutable “black box” AI systems. Through an iterative design process guided by stakeholder feedback, our multidisciplinary design team developed prototypes that were evaluated by experienced intelligence analysts. Results substantially supported the viability of PADTHAI-MM in designing for system trustworthiness in this task domain. We also explored the relationship between analysts' MAST ratings and three theoretical categories of information known to impact trust: process, purpose, and performance. Overall, our study supports the practical and theoretical viability of PADTHAI-MM as an approach to designing trustable AI systems.