ATR评价中绩效指标置信区间估计的样本量分析

Jun He, Hongzhong Zhao, Q. Fu
{"title":"ATR评价中绩效指标置信区间估计的样本量分析","authors":"Jun He, Hongzhong Zhao, Q. Fu","doi":"10.1109/RADAR.2007.374284","DOIUrl":null,"url":null,"abstract":"In this paper, we address the problem of sample size requirement for confidence interval estimation of performance metrics in ATR evaluation. The sample size of test data in ATR evaluation application has been reviewed firstly. We choose the Bayesian method to analyze the problem and select \"minimum length criterion\" to obtain the shortest confidence interval (CI) with the same estimation accuracy. The binomial probability density function is regarded as the likelihood function to calculate the posterior distribution about the metrics. Then CI accuracy requirement in ATR evaluation is discussed. Because the posterior distribution of the metrics depends on the test result, criteria to eliminate the uncertainty from the test result must be considered. The worst outcome (WOC) criterion is chosen to calculate sample sizes for various CI accuracy requirements and the corresponding sample sizes are listed in Table I. Table I shows that the sample size is very large when the CI accuracy requirement is high. Two approaches (specifications and prior information) to reduce the sample size are proposed and discussed. The absolute number of the sample size reduction is great when using the specifications in ATR evaluation applications. Table II contains those minimum sample sizes with various specifications, and its comparison with table I is shown. Whereas the sample size reduction is not large when using prior information because precise prior information about the metrics is always absent. The approximate sample size reductions can be got from table III and IV when considering beta distribution as the prior distribution.","PeriodicalId":367078,"journal":{"name":"2007 IEEE Radar Conference","volume":"241 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2007-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":"{\"title\":\"Sample Size Analysis for Confidence Interval Estimation of Performance Metrics in ATR Evaluation\",\"authors\":\"Jun He, Hongzhong Zhao, Q. Fu\",\"doi\":\"10.1109/RADAR.2007.374284\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In this paper, we address the problem of sample size requirement for confidence interval estimation of performance metrics in ATR evaluation. The sample size of test data in ATR evaluation application has been reviewed firstly. We choose the Bayesian method to analyze the problem and select \\\"minimum length criterion\\\" to obtain the shortest confidence interval (CI) with the same estimation accuracy. The binomial probability density function is regarded as the likelihood function to calculate the posterior distribution about the metrics. Then CI accuracy requirement in ATR evaluation is discussed. Because the posterior distribution of the metrics depends on the test result, criteria to eliminate the uncertainty from the test result must be considered. The worst outcome (WOC) criterion is chosen to calculate sample sizes for various CI accuracy requirements and the corresponding sample sizes are listed in Table I. Table I shows that the sample size is very large when the CI accuracy requirement is high. Two approaches (specifications and prior information) to reduce the sample size are proposed and discussed. The absolute number of the sample size reduction is great when using the specifications in ATR evaluation applications. Table II contains those minimum sample sizes with various specifications, and its comparison with table I is shown. Whereas the sample size reduction is not large when using prior information because precise prior information about the metrics is always absent. The approximate sample size reductions can be got from table III and IV when considering beta distribution as the prior distribution.\",\"PeriodicalId\":367078,\"journal\":{\"name\":\"2007 IEEE Radar Conference\",\"volume\":\"241 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2007-04-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"4\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2007 IEEE Radar Conference\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/RADAR.2007.374284\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2007 IEEE Radar Conference","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/RADAR.2007.374284","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 4

摘要

本文研究了ATR评价中性能指标置信区间估计的样本量要求问题。首先对ATR评价应用中测试数据的样本量进行了综述。我们选择贝叶斯方法对问题进行分析,选择“最小长度准则”来获得具有相同估计精度的最短置信区间(CI)。以二项概率密度函数作为似然函数,计算指标的后验分布。然后讨论了ATR评价中CI精度要求。由于指标的后验分布取决于测试结果,因此必须考虑消除测试结果不确定性的准则。选择最差结果(WOC)标准计算各种CI精度要求的样本量,相应的样本量列于表1。表1显示,当CI精度要求较高时,样本量非常大。两种方法(规格和先验信息),以减少样本量提出和讨论。当在ATR评估应用中使用规范时,样本量减少的绝对数量是很大的。表2包含了各种规格的最小样本量,并与表1进行了比较。然而,当使用先验信息时,样本量减少并不大,因为关于度量的精确先验信息总是缺失的。当考虑beta分布作为先验分布时,可以从表III和表IV中得到大致的样本量缩减。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Sample Size Analysis for Confidence Interval Estimation of Performance Metrics in ATR Evaluation
In this paper, we address the problem of sample size requirement for confidence interval estimation of performance metrics in ATR evaluation. The sample size of test data in ATR evaluation application has been reviewed firstly. We choose the Bayesian method to analyze the problem and select "minimum length criterion" to obtain the shortest confidence interval (CI) with the same estimation accuracy. The binomial probability density function is regarded as the likelihood function to calculate the posterior distribution about the metrics. Then CI accuracy requirement in ATR evaluation is discussed. Because the posterior distribution of the metrics depends on the test result, criteria to eliminate the uncertainty from the test result must be considered. The worst outcome (WOC) criterion is chosen to calculate sample sizes for various CI accuracy requirements and the corresponding sample sizes are listed in Table I. Table I shows that the sample size is very large when the CI accuracy requirement is high. Two approaches (specifications and prior information) to reduce the sample size are proposed and discussed. The absolute number of the sample size reduction is great when using the specifications in ATR evaluation applications. Table II contains those minimum sample sizes with various specifications, and its comparison with table I is shown. Whereas the sample size reduction is not large when using prior information because precise prior information about the metrics is always absent. The approximate sample size reductions can be got from table III and IV when considering beta distribution as the prior distribution.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信