水流时间序列中非自然记录的视觉检测:挑战与影响

IF 5.7 1区 地球科学 Q1 GEOSCIENCES, MULTIDISCIPLINARY
Laurent Strohmenger, Eric Sauquet, Claire Bernard, Jérémie Bonneau, Flora Branger, Amélie Bresson, Pierre Brigode, Rémy Buzier, Olivier Delaigue, Alexandre Devers, Guillaume Evin, Maïté Fournier, Shu-Chen Hsu, Sandra Lanini, Alban de Lavenne, Thibault Lemaitre-Basset, Claire Magand, Guilherme Mendoza Guimarães, Max Mentha, Simon Munier, Charles Perrin, Tristan Podechard, Léo Rouchy, Malak Sadki, Myriam Soutif-Bellenger, François Tilmant, Yves Tramblay, Anne-Lise Véron, Jean-Philippe Vidal, Guillaume Thirel
{"title":"水流时间序列中非自然记录的视觉检测:挑战与影响","authors":"Laurent Strohmenger, Eric Sauquet, Claire Bernard, Jérémie Bonneau, Flora Branger, Amélie Bresson, Pierre Brigode, Rémy Buzier, Olivier Delaigue, Alexandre Devers, Guillaume Evin, Maïté Fournier, Shu-Chen Hsu, Sandra Lanini, Alban de Lavenne, Thibault Lemaitre-Basset, Claire Magand, Guilherme Mendoza Guimarães, Max Mentha, Simon Munier, Charles Perrin, Tristan Podechard, Léo Rouchy, Malak Sadki, Myriam Soutif-Bellenger, François Tilmant, Yves Tramblay, Anne-Lise Véron, Jean-Philippe Vidal, Guillaume Thirel","doi":"10.5194/hess-27-3375-2023","DOIUrl":null,"url":null,"abstract":"Abstract. Large datasets of long-term streamflow measurements are widely used to infer and model hydrological processes. However, streamflow measurements may suffer from what users can consider anomalies, i.e. non-natural records that may be erroneous streamflow values or anthropogenic influences that can lead to misinterpretation of actual hydrological processes. Since identifying anomalies is time consuming for humans, no study has investigated their proportion, temporal distribution, and influence on hydrological indicators over large datasets. This study summarizes the results of a large visual inspection campaign of 674 streamflow time series in France made by 43 evaluators, who were asked to identify anomalies falling under five categories, namely, linear interpolation, drops, noise, point anomalies, and other. We examined the evaluators' individual behaviour in terms of severity and agreement with other evaluators, as well as the temporal distributions of the anomalies and their influence on commonly used hydrological indicators. We found that inter-evaluator agreement was surprisingly low, with an average of 12 % of overlapping periods reported as anomalies. These anomalies were mostly identified as linear interpolation and noise, and they were more frequently reported during the low-flow periods in summer. The impact of cleaning data from the identified anomaly values was higher on low-flow indicators than on high-flow indicators, with change rates lower than 5 % most of the time. We conclude that the identification of anomalies in streamflow time series is highly dependent on the aims and skills of each evaluator, which raises questions about the best practices to adopt for data cleaning.","PeriodicalId":13143,"journal":{"name":"Hydrology and Earth System Sciences","volume":"141 1","pages":"0"},"PeriodicalIF":5.7000,"publicationDate":"2023-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"On the visual detection of non-natural records in streamflow time series: challenges and impacts\",\"authors\":\"Laurent Strohmenger, Eric Sauquet, Claire Bernard, Jérémie Bonneau, Flora Branger, Amélie Bresson, Pierre Brigode, Rémy Buzier, Olivier Delaigue, Alexandre Devers, Guillaume Evin, Maïté Fournier, Shu-Chen Hsu, Sandra Lanini, Alban de Lavenne, Thibault Lemaitre-Basset, Claire Magand, Guilherme Mendoza Guimarães, Max Mentha, Simon Munier, Charles Perrin, Tristan Podechard, Léo Rouchy, Malak Sadki, Myriam Soutif-Bellenger, François Tilmant, Yves Tramblay, Anne-Lise Véron, Jean-Philippe Vidal, Guillaume Thirel\",\"doi\":\"10.5194/hess-27-3375-2023\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Abstract. Large datasets of long-term streamflow measurements are widely used to infer and model hydrological processes. However, streamflow measurements may suffer from what users can consider anomalies, i.e. non-natural records that may be erroneous streamflow values or anthropogenic influences that can lead to misinterpretation of actual hydrological processes. Since identifying anomalies is time consuming for humans, no study has investigated their proportion, temporal distribution, and influence on hydrological indicators over large datasets. This study summarizes the results of a large visual inspection campaign of 674 streamflow time series in France made by 43 evaluators, who were asked to identify anomalies falling under five categories, namely, linear interpolation, drops, noise, point anomalies, and other. We examined the evaluators' individual behaviour in terms of severity and agreement with other evaluators, as well as the temporal distributions of the anomalies and their influence on commonly used hydrological indicators. We found that inter-evaluator agreement was surprisingly low, with an average of 12 % of overlapping periods reported as anomalies. These anomalies were mostly identified as linear interpolation and noise, and they were more frequently reported during the low-flow periods in summer. The impact of cleaning data from the identified anomaly values was higher on low-flow indicators than on high-flow indicators, with change rates lower than 5 % most of the time. We conclude that the identification of anomalies in streamflow time series is highly dependent on the aims and skills of each evaluator, which raises questions about the best practices to adopt for data cleaning.\",\"PeriodicalId\":13143,\"journal\":{\"name\":\"Hydrology and Earth System Sciences\",\"volume\":\"141 1\",\"pages\":\"0\"},\"PeriodicalIF\":5.7000,\"publicationDate\":\"2023-09-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Hydrology and Earth System Sciences\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.5194/hess-27-3375-2023\",\"RegionNum\":1,\"RegionCategory\":\"地球科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"GEOSCIENCES, MULTIDISCIPLINARY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Hydrology and Earth System Sciences","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.5194/hess-27-3375-2023","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"GEOSCIENCES, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

摘要

摘要长期流量测量的大型数据集被广泛用于推断和模拟水文过程。然而,流量测量可能会受到用户认为异常的影响,即可能是错误的流量值的非自然记录或可能导致对实际水文过程的错误解释的人为影响。由于识别异常对人类来说是耗时的,因此没有研究在大型数据集上调查异常的比例、时间分布以及对水文指标的影响。本研究总结了由43名评估者在法国对674个水流时间序列进行的大型目视检查活动的结果,他们被要求识别五类异常,即线性插值、下降、噪声、点异常等。我们从严重程度和与其他评估人员的一致性方面检查了评估人员的个人行为,以及异常的时间分布及其对常用水文指标的影响。我们发现评估者之间的一致性出奇地低,平均有12%的重叠期被报告为异常。这些异常以线性插值和噪声为主,在夏季低流期更为常见。来自已识别异常值的清洗数据对低流量指标的影响大于对高流量指标的影响,大多数时间的变化率低于5%。我们得出的结论是,流时间序列中异常的识别高度依赖于每个评估者的目标和技能,这就提出了关于采用数据清理的最佳实践的问题。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
On the visual detection of non-natural records in streamflow time series: challenges and impacts
Abstract. Large datasets of long-term streamflow measurements are widely used to infer and model hydrological processes. However, streamflow measurements may suffer from what users can consider anomalies, i.e. non-natural records that may be erroneous streamflow values or anthropogenic influences that can lead to misinterpretation of actual hydrological processes. Since identifying anomalies is time consuming for humans, no study has investigated their proportion, temporal distribution, and influence on hydrological indicators over large datasets. This study summarizes the results of a large visual inspection campaign of 674 streamflow time series in France made by 43 evaluators, who were asked to identify anomalies falling under five categories, namely, linear interpolation, drops, noise, point anomalies, and other. We examined the evaluators' individual behaviour in terms of severity and agreement with other evaluators, as well as the temporal distributions of the anomalies and their influence on commonly used hydrological indicators. We found that inter-evaluator agreement was surprisingly low, with an average of 12 % of overlapping periods reported as anomalies. These anomalies were mostly identified as linear interpolation and noise, and they were more frequently reported during the low-flow periods in summer. The impact of cleaning data from the identified anomaly values was higher on low-flow indicators than on high-flow indicators, with change rates lower than 5 % most of the time. We conclude that the identification of anomalies in streamflow time series is highly dependent on the aims and skills of each evaluator, which raises questions about the best practices to adopt for data cleaning.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Hydrology and Earth System Sciences
Hydrology and Earth System Sciences 地学-地球科学综合
CiteScore
10.10
自引率
7.90%
发文量
273
审稿时长
15 months
期刊介绍: Hydrology and Earth System Sciences (HESS) is a not-for-profit international two-stage open-access journal for the publication of original research in hydrology. HESS encourages and supports fundamental and applied research that advances the understanding of hydrological systems, their role in providing water for ecosystems and society, and the role of the water cycle in the functioning of the Earth system. A multi-disciplinary approach is encouraged that broadens the hydrological perspective and the advancement of hydrological science through integration with other cognate sciences and cross-fertilization across disciplinary boundaries.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信