大规模数据流在线监测的自适应方法

IF 2 3区 工程技术 Q3 ENGINEERING, INDUSTRIAL
Shuchen Cao, Ruizhi Zhang
{"title":"大规模数据流在线监测的自适应方法","authors":"Shuchen Cao, Ruizhi Zhang","doi":"10.1080/24725854.2023.2281580","DOIUrl":null,"url":null,"abstract":"AbstractIn this paper, we propose an adaptive top-r method to monitor large-scale data streams where the change may affect a set of unknown data streams at some unknown time. Motivated by parallel and distributed computing, we propose to develop global monitoring schemes by parallel running local detection procedures and then use the Benjamin-Hochberg (BH) false discovery rate (FDR) control procedure to estimate the number of changed data streams adaptively. Our approach is illustrated in two concrete examples: one is a homogeneous case when all data streams are i.i.d with the same known pre-change and post-change distributions. The other is when all data are normally distributed, and the mean shifts are unknown and can be positive or negative. Theoretically, we show that when the pre-change and post-change distributions are completely specified, our proposed method can estimate the number of changed data streams for both the pre-change and post-change status. Moreover, we perform simulations and two case studies to show its detection efficiency.Keywords: False discovery rateCUSUMquickest change detectionprocess controlDisclaimerAs a service to authors and researchers we are providing this version of an accepted manuscript (AM). Copyediting, typesetting, and review of the resulting proofs will be undertaken on this manuscript before final publication of the Version of Record (VoR). During production and pre-press, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal relate to these versions also.","PeriodicalId":56039,"journal":{"name":"IISE Transactions","volume":null,"pages":null},"PeriodicalIF":2.0000,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"An Adaptive Approach for Online Monitoring of Large Scale Data Streams\",\"authors\":\"Shuchen Cao, Ruizhi Zhang\",\"doi\":\"10.1080/24725854.2023.2281580\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"AbstractIn this paper, we propose an adaptive top-r method to monitor large-scale data streams where the change may affect a set of unknown data streams at some unknown time. Motivated by parallel and distributed computing, we propose to develop global monitoring schemes by parallel running local detection procedures and then use the Benjamin-Hochberg (BH) false discovery rate (FDR) control procedure to estimate the number of changed data streams adaptively. Our approach is illustrated in two concrete examples: one is a homogeneous case when all data streams are i.i.d with the same known pre-change and post-change distributions. The other is when all data are normally distributed, and the mean shifts are unknown and can be positive or negative. Theoretically, we show that when the pre-change and post-change distributions are completely specified, our proposed method can estimate the number of changed data streams for both the pre-change and post-change status. Moreover, we perform simulations and two case studies to show its detection efficiency.Keywords: False discovery rateCUSUMquickest change detectionprocess controlDisclaimerAs a service to authors and researchers we are providing this version of an accepted manuscript (AM). Copyediting, typesetting, and review of the resulting proofs will be undertaken on this manuscript before final publication of the Version of Record (VoR). During production and pre-press, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal relate to these versions also.\",\"PeriodicalId\":56039,\"journal\":{\"name\":\"IISE Transactions\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.0000,\"publicationDate\":\"2023-11-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IISE Transactions\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1080/24725854.2023.2281580\",\"RegionNum\":3,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"ENGINEERING, INDUSTRIAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IISE Transactions","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/24725854.2023.2281580","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ENGINEERING, INDUSTRIAL","Score":null,"Total":0}
引用次数: 0

摘要

在本文中,我们提出了一种自适应top-r方法来监控大规模数据流,其中变化可能在某个未知时间影响一组未知数据流。在并行和分布式计算的驱动下,我们提出了通过并行运行局部检测程序来开发全局监测方案,然后使用Benjamin-Hochberg (BH)错误发现率(FDR)控制程序自适应估计变化数据流的数量。我们的方法用两个具体的例子来说明:一个是同质的情况,即所有数据流都具有相同的已知变化前和变化后的分布。另一种情况是,所有数据都是正态分布,平均位移是未知的,可以是正的,也可以是负的。从理论上讲,我们表明,当变化前和变化后的分布完全指定时,我们提出的方法可以估计变化前和变化后状态的数据流的数量。此外,我们还进行了仿真和两个案例研究,以证明其检测效率。关键词:错误发现率usum最快变化检测过程控制免责声明作为对作者和研究人员的服务,我们提供此版本的已接受稿件(AM)。在最终出版版本记录(VoR)之前,将对该手稿进行编辑、排版和审查。在制作和印前,可能会发现可能影响内容的错误,所有适用于期刊的法律免责声明也与这些版本有关。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
An Adaptive Approach for Online Monitoring of Large Scale Data Streams
AbstractIn this paper, we propose an adaptive top-r method to monitor large-scale data streams where the change may affect a set of unknown data streams at some unknown time. Motivated by parallel and distributed computing, we propose to develop global monitoring schemes by parallel running local detection procedures and then use the Benjamin-Hochberg (BH) false discovery rate (FDR) control procedure to estimate the number of changed data streams adaptively. Our approach is illustrated in two concrete examples: one is a homogeneous case when all data streams are i.i.d with the same known pre-change and post-change distributions. The other is when all data are normally distributed, and the mean shifts are unknown and can be positive or negative. Theoretically, we show that when the pre-change and post-change distributions are completely specified, our proposed method can estimate the number of changed data streams for both the pre-change and post-change status. Moreover, we perform simulations and two case studies to show its detection efficiency.Keywords: False discovery rateCUSUMquickest change detectionprocess controlDisclaimerAs a service to authors and researchers we are providing this version of an accepted manuscript (AM). Copyediting, typesetting, and review of the resulting proofs will be undertaken on this manuscript before final publication of the Version of Record (VoR). During production and pre-press, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal relate to these versions also.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IISE Transactions
IISE Transactions Engineering-Industrial and Manufacturing Engineering
CiteScore
5.70
自引率
7.70%
发文量
93
期刊介绍: IISE Transactions is currently abstracted/indexed in the following services: CSA/ASCE Civil Engineering Abstracts; CSA-Computer & Information Systems Abstracts; CSA-Corrosion Abstracts; CSA-Electronics & Communications Abstracts; CSA-Engineered Materials Abstracts; CSA-Materials Research Database with METADEX; CSA-Mechanical & Transportation Engineering Abstracts; CSA-Solid State & Superconductivity Abstracts; INSPEC Information Services and Science Citation Index. Institute of Industrial and Systems Engineers and our publisher Taylor & Francis make every effort to ensure the accuracy of all the information (the "Content") contained in our publications. However, Institute of Industrial and Systems Engineers and our publisher Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Institute of Industrial and Systems Engineers and our publisher Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Institute of Industrial and Systems Engineers and our publisher Taylor & Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to, or arising out of the use of the Content. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions .
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信