{"title":"海洋传感器数据流的概念漂移自适应异常检测器","authors":"Ngoc-Thanh Nguyen , Rogardt Heldal , Patrizio Pelliccione","doi":"10.1016/j.iot.2024.101414","DOIUrl":null,"url":null,"abstract":"<div><div>There is an increasing industry demand for reliable Anomaly Detection (AD) solutions to detect erroneous measurements or alarm events in marine sensor data. We evaluated 36 state-of-the-art AD algorithms published in the last three decades on our three real-world univariate time-series datasets collected from marine sensors. None of them achieved the accuracy expected from the marine industry. The algorithms are underperforming on our data because they are too generalized and cannot handle unforeseen data distribution changes, referred to as concept drift.</div><div>To address these issues, we developed a novel algorithm called <span>AdapAD</span>, which incorporates sensor design information into decision-making processes and can adapt to concept drift. We actively collaborated with nine domain experts from six marine organizations to ensure that <span>AdapAD</span> meets their needs. The experiments show that <span>AdapAD</span> satisfies the accuracy expectation and outperforms 40 existing AD algorithms. The decision-making time of <span>AdapAD</span>, measured on a commodity laptop, is generally less than one minute, showing its potential for application to perform real-time AD in marine sensor data streams. <span>AdapAD</span> was acknowledged as a viable solution for automatic marine data quality control and flood detection by 17 domain experts from 12 organizations. For transparency and to facilitate further research, we provide the implementation of <span>AdapAD</span> and the real-world marine sensor data used in the study.</div></div>","PeriodicalId":29968,"journal":{"name":"Internet of Things","volume":null,"pages":null},"PeriodicalIF":6.0000,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Concept-drift-adaptive anomaly detector for marine sensor data streams\",\"authors\":\"Ngoc-Thanh Nguyen , Rogardt Heldal , Patrizio Pelliccione\",\"doi\":\"10.1016/j.iot.2024.101414\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>There is an increasing industry demand for reliable Anomaly Detection (AD) solutions to detect erroneous measurements or alarm events in marine sensor data. We evaluated 36 state-of-the-art AD algorithms published in the last three decades on our three real-world univariate time-series datasets collected from marine sensors. None of them achieved the accuracy expected from the marine industry. The algorithms are underperforming on our data because they are too generalized and cannot handle unforeseen data distribution changes, referred to as concept drift.</div><div>To address these issues, we developed a novel algorithm called <span>AdapAD</span>, which incorporates sensor design information into decision-making processes and can adapt to concept drift. We actively collaborated with nine domain experts from six marine organizations to ensure that <span>AdapAD</span> meets their needs. The experiments show that <span>AdapAD</span> satisfies the accuracy expectation and outperforms 40 existing AD algorithms. The decision-making time of <span>AdapAD</span>, measured on a commodity laptop, is generally less than one minute, showing its potential for application to perform real-time AD in marine sensor data streams. <span>AdapAD</span> was acknowledged as a viable solution for automatic marine data quality control and flood detection by 17 domain experts from 12 organizations. For transparency and to facilitate further research, we provide the implementation of <span>AdapAD</span> and the real-world marine sensor data used in the study.</div></div>\",\"PeriodicalId\":29968,\"journal\":{\"name\":\"Internet of Things\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":6.0000,\"publicationDate\":\"2024-10-29\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Internet of Things\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S254266052400355X\",\"RegionNum\":3,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Internet of Things","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S254266052400355X","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
摘要
业界对可靠的异常检测(AD)解决方案的需求与日俱增,以检测海洋传感器数据中的错误测量或警报事件。我们在从海洋传感器收集的三个真实世界单变量时间序列数据集上评估了过去三十年中发布的 36 种最先进的 AD 算法。结果发现,没有一种算法能达到海洋行业的预期精度。这些算法在我们的数据上表现不佳,因为它们过于泛化,无法处理不可预见的数据分布变化,即概念漂移。为了解决这些问题,我们开发了一种名为 AdapAD 的新型算法,它将传感器设计信息纳入决策过程,并能适应概念漂移。我们与来自六个海洋组织的九位领域专家积极合作,确保 AdapAD 满足他们的需求。实验结果表明,AdapAD 达到了预期精度,并优于 40 种现有的 AD 算法。在商用笔记本电脑上测量的 AdapAD 决策时间通常少于一分钟,这表明它具有在海洋传感器数据流中执行实时 AD 的应用潜力。来自 12 个组织的 17 位领域专家认为 AdapAD 是自动海洋数据质量控制和洪水探测的可行解决方案。为了提高透明度和促进进一步研究,我们提供了 AdapAD 的实现方法和研究中使用的真实世界海洋传感器数据。
Concept-drift-adaptive anomaly detector for marine sensor data streams
There is an increasing industry demand for reliable Anomaly Detection (AD) solutions to detect erroneous measurements or alarm events in marine sensor data. We evaluated 36 state-of-the-art AD algorithms published in the last three decades on our three real-world univariate time-series datasets collected from marine sensors. None of them achieved the accuracy expected from the marine industry. The algorithms are underperforming on our data because they are too generalized and cannot handle unforeseen data distribution changes, referred to as concept drift.
To address these issues, we developed a novel algorithm called AdapAD, which incorporates sensor design information into decision-making processes and can adapt to concept drift. We actively collaborated with nine domain experts from six marine organizations to ensure that AdapAD meets their needs. The experiments show that AdapAD satisfies the accuracy expectation and outperforms 40 existing AD algorithms. The decision-making time of AdapAD, measured on a commodity laptop, is generally less than one minute, showing its potential for application to perform real-time AD in marine sensor data streams. AdapAD was acknowledged as a viable solution for automatic marine data quality control and flood detection by 17 domain experts from 12 organizations. For transparency and to facilitate further research, we provide the implementation of AdapAD and the real-world marine sensor data used in the study.
期刊介绍:
Internet of Things; Engineering Cyber Physical Human Systems is a comprehensive journal encouraging cross collaboration between researchers, engineers and practitioners in the field of IoT & Cyber Physical Human Systems. The journal offers a unique platform to exchange scientific information on the entire breadth of technology, science, and societal applications of the IoT.
The journal will place a high priority on timely publication, and provide a home for high quality.
Furthermore, IOT is interested in publishing topical Special Issues on any aspect of IOT.