The Troublesome Kernel: On Hallucinations, No Free Lunches, and the Accuracy-Stability Tradeoff in Inverse Problems

IF 10.8 1区 数学 Q1 MATHEMATICS, APPLIED
SIAM Review Pub Date : 2025-02-06 DOI:10.1137/23m1568739
Nina M. Gottschling, Vegard Antun, Anders C. Hansen, Ben Adcock
{"title":"The Troublesome Kernel: On Hallucinations, No Free Lunches, and the Accuracy-Stability Tradeoff in Inverse Problems","authors":"Nina M. Gottschling, Vegard Antun, Anders C. Hansen, Ben Adcock","doi":"10.1137/23m1568739","DOIUrl":null,"url":null,"abstract":"SIAM Review, Volume 67, Issue 1, Page 73-104, March 2025. <br/> Abstract.Methods inspired by artificial intelligence (AI) are starting to fundamentally change computational science and engineering through breakthrough performance on challenging problems. However, the reliability and trustworthiness of such techniques is a major concern. In inverse problems in imaging, the focus of this paper, there is increasing empirical evidence that methods may suffer from hallucinations, i.e., false, but realistic-looking artifacts; instability, i.e., sensitivity to perturbations in the data; and unpredictable generalization, i.e., excellent performance on some images, but significant deterioration on others. This paper provides a theoretical foundation for these phenomena. We give mathematical explanations for how and when such effects arise in arbitrary reconstruction methods, with several of our results taking the form of “no free lunch” theorems. Specifically, we show that (i) methods that overperform on a single image can wrongly transfer details from one image to another, creating a hallucination; (ii) methods that overperform on two or more images can hallucinate or be unstable; (iii) optimizing the accuracy-stability tradeoff is generally difficult; (iv) hallucinations and instabilities, if they occur, are not rare events and may be encouraged by standard training; and (v) it may be impossible to construct optimal reconstruction maps for certain problems. Our results trace these effects to the kernel of the forward operator whenever it is nontrivial, but also apply to the case when the forward operator is ill-conditioned. Based on these insights, our work aims to spur research into new ways to develop robust and reliable AI-based methods for inverse problems in imaging.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"123 1","pages":""},"PeriodicalIF":10.8000,"publicationDate":"2025-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"SIAM Review","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1137/23m1568739","RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}
引用次数: 0

Abstract

SIAM Review, Volume 67, Issue 1, Page 73-104, March 2025.
Abstract.Methods inspired by artificial intelligence (AI) are starting to fundamentally change computational science and engineering through breakthrough performance on challenging problems. However, the reliability and trustworthiness of such techniques is a major concern. In inverse problems in imaging, the focus of this paper, there is increasing empirical evidence that methods may suffer from hallucinations, i.e., false, but realistic-looking artifacts; instability, i.e., sensitivity to perturbations in the data; and unpredictable generalization, i.e., excellent performance on some images, but significant deterioration on others. This paper provides a theoretical foundation for these phenomena. We give mathematical explanations for how and when such effects arise in arbitrary reconstruction methods, with several of our results taking the form of “no free lunch” theorems. Specifically, we show that (i) methods that overperform on a single image can wrongly transfer details from one image to another, creating a hallucination; (ii) methods that overperform on two or more images can hallucinate or be unstable; (iii) optimizing the accuracy-stability tradeoff is generally difficult; (iv) hallucinations and instabilities, if they occur, are not rare events and may be encouraged by standard training; and (v) it may be impossible to construct optimal reconstruction maps for certain problems. Our results trace these effects to the kernel of the forward operator whenever it is nontrivial, but also apply to the case when the forward operator is ill-conditioned. Based on these insights, our work aims to spur research into new ways to develop robust and reliable AI-based methods for inverse problems in imaging.
麻烦的核:关于幻觉,没有免费的午餐,以及反问题的精度-稳定性权衡
SIAM评论,第67卷,第1期,第73-104页,2025年3月。摘要。受人工智能(AI)启发的方法通过在挑战性问题上的突破性表现,开始从根本上改变计算科学和工程。然而,这些技术的可靠性和可信赖性是一个主要问题。在成像的逆问题中,本文的重点,有越来越多的经验证据表明,方法可能会产生幻觉,即虚假的,但看起来很逼真的工件;不稳定性,即对数据扰动的敏感性;不可预测的泛化,即在某些图像上表现出色,但在其他图像上明显恶化。本文为这些现象提供了理论依据。我们给出了在任意重建方法中如何以及何时出现这种效应的数学解释,我们的一些结果采用了“没有免费的午餐”定理的形式。具体来说,我们表明(i)在单张图像上表现过度的方法可能会错误地将细节从一张图像转移到另一张图像,从而产生幻觉;(ii)在两个或多个图像上表现过度的方法可能会产生幻觉或不稳定;(iii)优化精度与稳定性的权衡通常是困难的;(iv)幻觉和不稳定,如果发生,不是罕见事件,可以通过标准训练加以鼓励;(5)对于某些问题,可能不可能构造出最优的重构图。我们的结果将这些影响追溯到正向运算符的核,只要它是非平凡的,但也适用于正向运算符是病态的情况。基于这些见解,我们的工作旨在促进研究新方法,以开发稳健可靠的基于人工智能的成像逆问题方法。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
SIAM Review
SIAM Review 数学-应用数学
CiteScore
16.90
自引率
0.00%
发文量
50
期刊介绍: Survey and Review feature papers that provide an integrative and current viewpoint on important topics in applied or computational mathematics and scientific computing. These papers aim to offer a comprehensive perspective on the subject matter. Research Spotlights publish concise research papers in applied and computational mathematics that are of interest to a wide range of readers in SIAM Review. The papers in this section present innovative ideas that are clearly explained and motivated. They stand out from regular publications in specific SIAM journals due to their accessibility and potential for widespread and long-lasting influence.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信