Multilevel Stochastic Optimization for Imputation in Massive Medical Data Records

IF 7.5 3区计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS

IEEE Transactions on Big Data Pub Date : 2023-10-30 DOI:10.1109/TBDATA.2023.3328433

Wenrui Li;Xiaoyu Wang;Yuetian Sun;Snezana Milanovic;Mark Kon;Julio Enrique Castrillón-Candás

{"title":"Multilevel Stochastic Optimization for Imputation in Massive Medical Data Records","authors":"Wenrui Li;Xiaoyu Wang;Yuetian Sun;Snezana Milanovic;Mark Kon;Julio Enrique Castrillón-Candás","doi":"10.1109/TBDATA.2023.3328433","DOIUrl":null,"url":null,"abstract":"It has long been a recognized problem that many datasets contain significant levels of missing numerical data. A potentially critical predicate for application of machine learning methods to datasets involves addressing this problem. However, this is a challenging task. In this article, we apply a recently developed multi-level stochastic optimization approach to the problem of imputation in massive medical records. The approach is based on computational applied mathematics techniques and is highly accurate. In particular, for the Best Linear Unbiased Predictor (BLUP) this multi-level formulation is \n<italic>exact</i>\n, and is significantly faster and more numerically stable. This permits practical application of Kriging methods to data imputation problems for massive datasets. We test this approach on data from the National Inpatient Sample (NIS) data records, Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality. Numerical results show that the multi-level method significantly outperforms current approaches and is numerically robust. It has superior accuracy as compared with methods recommended in the recent report from HCUP. Benchmark tests show up to 75% reductions in error. Furthermore, the results are also superior to recent state of the art methods such as discriminative deep learning.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"10 2","pages":"122-131"},"PeriodicalIF":7.5000,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Big Data","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10301500/","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

It has long been a recognized problem that many datasets contain significant levels of missing numerical data. A potentially critical predicate for application of machine learning methods to datasets involves addressing this problem. However, this is a challenging task. In this article, we apply a recently developed multi-level stochastic optimization approach to the problem of imputation in massive medical records. The approach is based on computational applied mathematics techniques and is highly accurate. In particular, for the Best Linear Unbiased Predictor (BLUP) this multi-level formulation is exact , and is significantly faster and more numerically stable. This permits practical application of Kriging methods to data imputation problems for massive datasets. We test this approach on data from the National Inpatient Sample (NIS) data records, Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality. Numerical results show that the multi-level method significantly outperforms current approaches and is numerically robust. It has superior accuracy as compared with methods recommended in the recent report from HCUP. Benchmark tests show up to 75% reductions in error. Furthermore, the results are also superior to recent state of the art methods such as discriminative deep learning.

查看原文本刊更多论文

在海量医疗数据记录中进行多层次随机优化归算

长期以来，许多数据集包含大量缺失的数字数据，这是一个公认的问题。将机器学习方法应用于数据集的一个潜在关键前提就是解决这一问题。然而，这是一项具有挑战性的任务。在本文中，我们将最近开发的多层次随机优化方法应用于海量医疗记录的估算问题。该方法基于计算应用数学技术，具有很高的准确性。特别是，对于最佳线性无偏预测（BLUP），这种多层次公式是精确的，而且速度更快，数值更稳定。这使得克里金方法可以实际应用于海量数据集的数据估算问题。我们在全国住院病人抽样（NIS）数据记录、医疗保健成本与利用项目（HCUP）、医疗保健研究与质量局的数据上测试了这种方法。数值结果表明，多层次方法明显优于当前的方法，并且在数值上是稳健的。与 HCUP 近期报告中推荐的方法相比，它具有更高的准确性。基准测试表明，误差最多可减少 75%。此外，其结果也优于最近的先进方法，如判别式深度学习。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Big Data Multiple-

CiteScore

11.80

自引率

2.80%

发文量

114

期刊介绍： The IEEE Transactions on Big Data publishes peer-reviewed articles focusing on big data. These articles present innovative research ideas and application results across disciplines, including novel theories, algorithms, and applications. Research areas cover a wide range, such as big data analytics, visualization, curation, management, semantics, infrastructure, standards, performance analysis, intelligence extraction, scientific discovery, security, privacy, and legal issues specific to big data. The journal also prioritizes applications of big data in fields generating massive datasets.