Comprehension is a double-edged sword: Over-interpreting unspecified information in intelligible machine learning explanations

IF 5.3 2区 计算机科学 Q1 COMPUTER SCIENCE, CYBERNETICS
Yueqing Xuan , Edward Small , Kacper Sokol , Danula Hettiachchi , Mark Sanderson
{"title":"Comprehension is a double-edged sword: Over-interpreting unspecified information in intelligible machine learning explanations","authors":"Yueqing Xuan ,&nbsp;Edward Small ,&nbsp;Kacper Sokol ,&nbsp;Danula Hettiachchi ,&nbsp;Mark Sanderson","doi":"10.1016/j.ijhcs.2024.103376","DOIUrl":null,"url":null,"abstract":"<div><div>Automated decision-making systems are becoming increasingly ubiquitous, which creates an immediate need for their interpretability and explainability. However, it remains unclear whether users know what insights an explanation offers and, more importantly, what information it lacks. To answer this question we conducted an online study with 200 participants, which allowed us to assess explainees’ ability to realise <em>explicated information</em> – i.e., factual insights conveyed by an explanation – and <em>unspecified information</em> – i.e, insights that are not communicated by an explanation – across four representative explanation types: model architecture, decision surface visualisation, counterfactual explainability and feature importance. Our findings uncover that highly comprehensible explanations, e.g., feature importance and decision surface visualisation, are exceptionally susceptible to misinterpretation since users tend to infer spurious information that is outside of the scope of these explanations. Additionally, while the users gauge their confidence accurately with respect to the information explicated by these explanations, they tend to be overconfident when misinterpreting the explanations. Our work demonstrates that human comprehension can be a double-edged sword since highly accessible explanations may convince users of their truthfulness while possibly leading to various misinterpretations at the same time. Machine learning explanations should therefore carefully navigate the complex relation between their full scope and limitations to maximise understanding and curb misinterpretation.</div></div>","PeriodicalId":54955,"journal":{"name":"International Journal of Human-Computer Studies","volume":"193 ","pages":"Article 103376"},"PeriodicalIF":5.3000,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Human-Computer Studies","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1071581924001599","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, CYBERNETICS","Score":null,"Total":0}
引用次数: 0

Abstract

Automated decision-making systems are becoming increasingly ubiquitous, which creates an immediate need for their interpretability and explainability. However, it remains unclear whether users know what insights an explanation offers and, more importantly, what information it lacks. To answer this question we conducted an online study with 200 participants, which allowed us to assess explainees’ ability to realise explicated information – i.e., factual insights conveyed by an explanation – and unspecified information – i.e, insights that are not communicated by an explanation – across four representative explanation types: model architecture, decision surface visualisation, counterfactual explainability and feature importance. Our findings uncover that highly comprehensible explanations, e.g., feature importance and decision surface visualisation, are exceptionally susceptible to misinterpretation since users tend to infer spurious information that is outside of the scope of these explanations. Additionally, while the users gauge their confidence accurately with respect to the information explicated by these explanations, they tend to be overconfident when misinterpreting the explanations. Our work demonstrates that human comprehension can be a double-edged sword since highly accessible explanations may convince users of their truthfulness while possibly leading to various misinterpretations at the same time. Machine learning explanations should therefore carefully navigate the complex relation between their full scope and limitations to maximise understanding and curb misinterpretation.
理解是一把双刃剑:在可理解的机器学习解释中过度解读未指定信息
自动决策系统正变得越来越无处不在,这就产生了对其可解释性和可说明性的迫切需求。然而,用户是否知道解释提供了哪些见解,以及更重要的是,解释缺乏哪些信息,这一点仍不清楚。为了回答这个问题,我们进行了一项有 200 名参与者参加的在线研究,通过这项研究,我们可以评估被解释者在四种具有代表性的解释类型(模型架构、决策面可视化、反事实可解释性和特征重要性)中实现解释信息(即解释所传达的事实见解)和未指明信息(即解释未传达的见解)的能力。我们的研究结果发现,高度可理解的解释(如特征重要性和决策面可视化)特别容易被误解,因为用户倾向于推断出这些解释范围之外的虚假信息。此外,虽然用户能准确衡量自己对这些解释所阐述信息的信心,但在误读解释时往往会过于自信。我们的工作表明,人类的理解能力可能是一把双刃剑,因为高度易懂的解释可能会让用户相信其真实性,但同时也可能导致各种误解。因此,机器学习解释应谨慎处理其全部范围和局限性之间的复杂关系,以最大限度地提高理解力并遏制误读。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
International Journal of Human-Computer Studies
International Journal of Human-Computer Studies 工程技术-计算机:控制论
CiteScore
11.50
自引率
5.60%
发文量
108
审稿时长
3 months
期刊介绍: The International Journal of Human-Computer Studies publishes original research over the whole spectrum of work relevant to the theory and practice of innovative interactive systems. The journal is inherently interdisciplinary, covering research in computing, artificial intelligence, psychology, linguistics, communication, design, engineering, and social organization, which is relevant to the design, analysis, evaluation and application of innovative interactive systems. Papers at the boundaries of these disciplines are especially welcome, as it is our view that interdisciplinary approaches are needed for producing theoretical insights in this complex area and for effective deployment of innovative technologies in concrete user communities. Research areas relevant to the journal include, but are not limited to: • Innovative interaction techniques • Multimodal interaction • Speech interaction • Graphic interaction • Natural language interaction • Interaction in mobile and embedded systems • Interface design and evaluation methodologies • Design and evaluation of innovative interactive systems • User interface prototyping and management systems • Ubiquitous computing • Wearable computers • Pervasive computing • Affective computing • Empirical studies of user behaviour • Empirical studies of programming and software engineering • Computer supported cooperative work • Computer mediated communication • Virtual reality • Mixed and augmented Reality • Intelligent user interfaces • Presence ...
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信