A machine learning oracle for parameter estimation

IF 2.1 4区数学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Statistical Analysis and Data Mining Pub Date : 2023-12-09 DOI:10.1002/sam.11651

Lucas Koepke, Mary Gregg, Michael Frey

{"title":"A machine learning oracle for parameter estimation","authors":"Lucas Koepke, Mary Gregg, Michael Frey","doi":"10.1002/sam.11651","DOIUrl":null,"url":null,"abstract":"Competing procedures, involving data smoothing, weighting, imputation, outlier removal, etc., may be available to prepare data for parametric model estimation. Often, however, little is known about the best choice of preparatory procedure for the planned estimation and the observed data. A machine learning-based decision rule, an “oracle,” can be constructed in such cases to decide the best procedure from a set <math altimg=\"urn:x-wiley:19321864:media:sam11651:sam11651-math-0001\" display=\"inline\" location=\"graphic/sam11651-math-0001.png\" overflow=\"scroll\">\n<semantics>\n<mrow>\n<mi mathvariant=\"script\">C</mi>\n</mrow>\n$$ \\mathcal{C} $$</annotation>\n</semantics></math> of available preparatory procedures. The oracle learns the decision regions associated with <math altimg=\"urn:x-wiley:19321864:media:sam11651:sam11651-math-0002\" display=\"inline\" location=\"graphic/sam11651-math-0002.png\" overflow=\"scroll\">\n<semantics>\n<mrow>\n<mi mathvariant=\"script\">C</mi>\n</mrow>\n$$ \\mathcal{C} $$</annotation>\n</semantics></math> based on training data synthesized solely from the given data using model parameters with high posterior probability. An estimator in combination with an oracle to guide data preparation is called an oracle estimator. Oracle estimator performance is studied in two estimation problems: slope estimation in simple linear regression (SLR) and changepoint estimation in continuous two-linear-segments regression (CTLSR). In both examples, the regression response is given to be increasing, and the oracle must decide whether to isotonically smooth the response data preparatory to fitting the regression model. A measure of performance called headroom is proposed to assess the oracle's potential for reducing estimation error. Experiments with SLR and CTLSR find for important ranges of problem configurations that the headroom is high, the oracle's empirical performance is near the headroom, and the oracle estimator offers clear benefit.","PeriodicalId":48684,"journal":{"name":"Statistical Analysis and Data Mining","volume":"64 1","pages":""},"PeriodicalIF":2.1000,"publicationDate":"2023-12-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Statistical Analysis and Data Mining","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1002/sam.11651","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

Competing procedures, involving data smoothing, weighting, imputation, outlier removal, etc., may be available to prepare data for parametric model estimation. Often, however, little is known about the best choice of preparatory procedure for the planned estimation and the observed data. A machine learning-based decision rule, an “oracle,” can be constructed in such cases to decide the best procedure from a set

C $$ \mathcal{C} $$

of available preparatory procedures. The oracle learns the decision regions associated with

C $$ \mathcal{C} $$

based on training data synthesized solely from the given data using model parameters with high posterior probability. An estimator in combination with an oracle to guide data preparation is called an oracle estimator. Oracle estimator performance is studied in two estimation problems: slope estimation in simple linear regression (SLR) and changepoint estimation in continuous two-linear-segments regression (CTLSR). In both examples, the regression response is given to be increasing, and the oracle must decide whether to isotonically smooth the response data preparatory to fitting the regression model. A measure of performance called headroom is proposed to assess the oracle's potential for reducing estimation error. Experiments with SLR and CTLSR find for important ranges of problem configurations that the headroom is high, the oracle's empirical performance is near the headroom, and the oracle estimator offers clear benefit.

查看原文本刊更多论文

用于参数估计的机器学习算法

数据平滑、加权、估算、离群值剔除等竞争性程序可用于参数模型估算的数据准备。然而，对于计划估算和观测数据的最佳准备程序选择，人们往往知之甚少。在这种情况下，可以构建一个基于机器学习的决策规则，即 "oracle"，以便从可用准备程序集 C$$ \mathcal{C}$$ 中选出最佳程序。甲骨文根据仅从给定数据合成的训练数据，使用具有高后验概率的模型参数，学习与 C$$ \mathcal{C}$ 相关的决策区域。与指导数据准备的甲骨文相结合的估计器称为甲骨文估计器。甲骨文估计器的性能在两个估计问题中进行了研究：简单线性回归（SLR）中的斜率估计和连续双线段回归（CTLSR）中的变化点估计。在这两个例子中，给定的回归响应都是递增的，甲骨文必须决定是否在拟合回归模型之前对响应数据进行同调平滑。我们提出了一种称为 "余量"（headroom）的性能测量方法，用于评估神谕在减少估计误差方面的潜力。利用 SLR 和 CTLSR 进行的实验发现，在重要的问题配置范围内，余量很大，甲骨文的经验性能接近余量，而且甲骨文估计器具有明显的优势。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Statistical Analysis and Data Mining COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCEC-COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

CiteScore

3.20

自引率

7.70%

发文量

期刊介绍： Statistical Analysis and Data Mining addresses the broad area of data analysis, including statistical approaches, machine learning, data mining, and applications. Topics include statistical and computational approaches for analyzing massive and complex datasets, novel statistical and/or machine learning methods and theory, and state-of-the-art applications with high impact. Of special interest are articles that describe innovative analytical techniques, and discuss their application to real problems, in such a way that they are accessible and beneficial to domain experts across science, engineering, and commerce. The focus of the journal is on papers which satisfy one or more of the following criteria: Solve data analysis problems associated with massive, complex datasets Develop innovative statistical approaches, machine learning algorithms, or methods integrating ideas across disciplines, e.g., statistics, computer science, electrical engineering, operation research. Formulate and solve high-impact real-world problems which challenge existing paradigms via new statistical and/or computational models Provide survey to prominent research topics.