Quantifying Epistemic Uncertainty in Binary Classification via Accuracy Gain

IF 2.1 4区数学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Statistical Analysis and Data Mining Pub Date : 2024-09-18 DOI:10.1002/sam.11709

Christopher Qian, Tyler Ganter, Joshua Michalenko, Feng Liang, Jason Adams

引用次数: 0

Abstract

Recently, a surge of interest has been given to quantifying epistemic uncertainty (EU), the reducible portion of uncertainty due to lack of data. We propose a novel EU estimator in the binary classification setting, as the posterior expected value of the empirical gain in accuracy between the current prediction and the optimal prediction. In order to validate the performance of our EU estimator, we introduce an experimental procedure where we take an existing dataset, remove a set of points, and compare the estimated EU with the observed change in accuracy. Through real and simulated data experiments, we demonstrate the effectiveness of our proposed EU estimator.

查看原文本刊更多论文

通过精度增益量化二元分类中的认识不确定性

最近，人们对量化认识不确定性（EU）产生了浓厚的兴趣，认识不确定性是由于缺乏数据而产生的不确定性中可减少的部分。我们在二元分类设置中提出了一种新的 EU 估计器，即当前预测与最优预测之间准确性经验增益的后验期望值。为了验证我们的 EU 估计器的性能，我们引入了一个实验过程，即利用现有数据集，移除一组点，然后将估计的 EU 与观察到的准确率变化进行比较。通过真实和模拟数据实验，我们证明了我们提出的 EU 估计器的有效性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Statistical Analysis and Data Mining COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCEC-COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

CiteScore

3.20

自引率

7.70%

发文量

期刊介绍： Statistical Analysis and Data Mining addresses the broad area of data analysis, including statistical approaches, machine learning, data mining, and applications. Topics include statistical and computational approaches for analyzing massive and complex datasets, novel statistical and/or machine learning methods and theory, and state-of-the-art applications with high impact. Of special interest are articles that describe innovative analytical techniques, and discuss their application to real problems, in such a way that they are accessible and beneficial to domain experts across science, engineering, and commerce. The focus of the journal is on papers which satisfy one or more of the following criteria: Solve data analysis problems associated with massive, complex datasets Develop innovative statistical approaches, machine learning algorithms, or methods integrating ideas across disciplines, e.g., statistics, computer science, electrical engineering, operation research. Formulate and solve high-impact real-world problems which challenge existing paradigms via new statistical and/or computational models Provide survey to prominent research topics.