Concept-drift-adaptive anomaly detector for marine sensor data streams

IF 6 3区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS
Ngoc-Thanh Nguyen , Rogardt Heldal , Patrizio Pelliccione
{"title":"Concept-drift-adaptive anomaly detector for marine sensor data streams","authors":"Ngoc-Thanh Nguyen ,&nbsp;Rogardt Heldal ,&nbsp;Patrizio Pelliccione","doi":"10.1016/j.iot.2024.101414","DOIUrl":null,"url":null,"abstract":"<div><div>There is an increasing industry demand for reliable Anomaly Detection (AD) solutions to detect erroneous measurements or alarm events in marine sensor data. We evaluated 36 state-of-the-art AD algorithms published in the last three decades on our three real-world univariate time-series datasets collected from marine sensors. None of them achieved the accuracy expected from the marine industry. The algorithms are underperforming on our data because they are too generalized and cannot handle unforeseen data distribution changes, referred to as concept drift.</div><div>To address these issues, we developed a novel algorithm called <span>AdapAD</span>, which incorporates sensor design information into decision-making processes and can adapt to concept drift. We actively collaborated with nine domain experts from six marine organizations to ensure that <span>AdapAD</span> meets their needs. The experiments show that <span>AdapAD</span> satisfies the accuracy expectation and outperforms 40 existing AD algorithms. The decision-making time of <span>AdapAD</span>, measured on a commodity laptop, is generally less than one minute, showing its potential for application to perform real-time AD in marine sensor data streams. <span>AdapAD</span> was acknowledged as a viable solution for automatic marine data quality control and flood detection by 17 domain experts from 12 organizations. For transparency and to facilitate further research, we provide the implementation of <span>AdapAD</span> and the real-world marine sensor data used in the study.</div></div>","PeriodicalId":29968,"journal":{"name":"Internet of Things","volume":"28 ","pages":"Article 101414"},"PeriodicalIF":6.0000,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Internet of Things","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S254266052400355X","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

There is an increasing industry demand for reliable Anomaly Detection (AD) solutions to detect erroneous measurements or alarm events in marine sensor data. We evaluated 36 state-of-the-art AD algorithms published in the last three decades on our three real-world univariate time-series datasets collected from marine sensors. None of them achieved the accuracy expected from the marine industry. The algorithms are underperforming on our data because they are too generalized and cannot handle unforeseen data distribution changes, referred to as concept drift.
To address these issues, we developed a novel algorithm called AdapAD, which incorporates sensor design information into decision-making processes and can adapt to concept drift. We actively collaborated with nine domain experts from six marine organizations to ensure that AdapAD meets their needs. The experiments show that AdapAD satisfies the accuracy expectation and outperforms 40 existing AD algorithms. The decision-making time of AdapAD, measured on a commodity laptop, is generally less than one minute, showing its potential for application to perform real-time AD in marine sensor data streams. AdapAD was acknowledged as a viable solution for automatic marine data quality control and flood detection by 17 domain experts from 12 organizations. For transparency and to facilitate further research, we provide the implementation of AdapAD and the real-world marine sensor data used in the study.
海洋传感器数据流的概念漂移自适应异常检测器
业界对可靠的异常检测(AD)解决方案的需求与日俱增,以检测海洋传感器数据中的错误测量或警报事件。我们在从海洋传感器收集的三个真实世界单变量时间序列数据集上评估了过去三十年中发布的 36 种最先进的 AD 算法。结果发现,没有一种算法能达到海洋行业的预期精度。这些算法在我们的数据上表现不佳,因为它们过于泛化,无法处理不可预见的数据分布变化,即概念漂移。为了解决这些问题,我们开发了一种名为 AdapAD 的新型算法,它将传感器设计信息纳入决策过程,并能适应概念漂移。我们与来自六个海洋组织的九位领域专家积极合作,确保 AdapAD 满足他们的需求。实验结果表明,AdapAD 达到了预期精度,并优于 40 种现有的 AD 算法。在商用笔记本电脑上测量的 AdapAD 决策时间通常少于一分钟,这表明它具有在海洋传感器数据流中执行实时 AD 的应用潜力。来自 12 个组织的 17 位领域专家认为 AdapAD 是自动海洋数据质量控制和洪水探测的可行解决方案。为了提高透明度和促进进一步研究,我们提供了 AdapAD 的实现方法和研究中使用的真实世界海洋传感器数据。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Internet of Things
Internet of Things Multiple-
CiteScore
3.60
自引率
5.10%
发文量
115
审稿时长
37 days
期刊介绍: Internet of Things; Engineering Cyber Physical Human Systems is a comprehensive journal encouraging cross collaboration between researchers, engineers and practitioners in the field of IoT & Cyber Physical Human Systems. The journal offers a unique platform to exchange scientific information on the entire breadth of technology, science, and societal applications of the IoT. The journal will place a high priority on timely publication, and provide a home for high quality. Furthermore, IOT is interested in publishing topical Special Issues on any aspect of IOT.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信