User-Centered Evaluation of Explainable Artificial Intelligence (XAI): A Systematic Literature Review

Abstract

Researchers have developed a variety of approaches to evaluate explainable artificial intelligence (XAI) systems using human–computer interaction (HCI) user-centered techniques. This systematic literature review has been conducted to understand how these approaches are used to achieve XAI goals. The aim of this review is to explore the methods used to evaluate XAI systems in studies involving human subjects. A total of 101 full-text studies were systematically selected and analyzed from a sample of 3414 studies obtained from four renowned databases between 2018 and 2023. The analysis focuses on prominent XAI goals achieved across 10 domains and the machine learning (ML) models utilized to create these XAI systems. The analysis also explores explanation methods and detailed study methodologies used by researchers in previous work. The analysis is concluded by categorizing the challenges experienced by researchers into three types. Exploring the methodologies employed by researchers, the review discusses the benefits and shortcomings of the data collection methods and participant recruitment. In conclusion, this review offers a framework that consists of six pillars that researchers can follow for evaluating user-centered studies in the field of XAI.