用优化的人工输入大规模注释神经生理学数据。

IF 3.8
Zhongchuan Xu, Brittany H Scheid, Erin C Conrad, Kathryn A Davis, Taneeta Ganguly, Michael A Gelfand, James J Gugger, Xiangyu Jiang, Joshua J LaRocque, William K S Ojemann, Saurabh R Sinha, Genna J Waldman, Joost Wagenaar, Nishant Sinha, Brian Litt
{"title":"用优化的人工输入大规模注释神经生理学数据。","authors":"Zhongchuan Xu, Brittany H Scheid, Erin C Conrad, Kathryn A Davis, Taneeta Ganguly, Michael A Gelfand, James J Gugger, Xiangyu Jiang, Joshua J LaRocque, William K S Ojemann, Saurabh R Sinha, Genna J Waldman, Joost Wagenaar, Nishant Sinha, Brian Litt","doi":"10.1088/1741-2552/ade402","DOIUrl":null,"url":null,"abstract":"<p><p><i>Objective.</i>Neuroscience experiments and devices are generating unprecedented volumes of data, but analyzing and validating them presents practical challenges, particularly in annotation. While expert annotation remains the gold standard, it is time consuming to obtain and often poorly reproducible. Although automated annotation approaches exist, they rely on labeled data first to train machine learning algorithms, which limits their scalability. A semi-automated annotation approach that integrates human expertise while optimizing efficiency at scale is critically needed. To address this, we present Annotation Co-pilot, a human-in-the-loop solution that leverages deep active learning (AL) and self-supervised learning (SSL) to improve intracranial EEG (iEEG) annotation, significantly reducing the amount of human annotations.<i>Approach.</i>We automatically annotated iEEG recordings from 28 humans and 4 dogs with epilepsy implanted with two neurodevices that telemetered data to the cloud for analysis. We processed 1500 h of unlabeled iEEG recordings to train a deep neural network using a SSL method Swapping Assignments between View to generate robust, dataset-specific feature embeddings for the purpose of seizure detection. AL was used to select only the most informative data epochs for expert review. We benchmarked this strategy against standard methods.<i>Main result.</i>Over 80 000 iEEG clips, totaling 1176 h of recordings were analyzed. The algorithm matched the best published seizure detectors on two datasets (NeuroVista and NeuroPace responsive neurostimulation) but required, on average, only 1/6 of the human annotations to achieve similar accuracy (area under the ROC curve of 0.9628 ± 0.015) and demonstrated better consistency than human annotators (Cohen's Kappa of 0.95 ± 0.04).<i>Significance</i>. 'Annotation Co-pilot' demonstrated expert-level performance, robustness, and generalizability across two disparate iEEG datasets while reducing annotation time by an average of 83%. This method holds great promise for accelerating basic and translational research in electrophysiology, and potentially accelerating the pathway to clinical translation for AI-based algorithms and devices.</p>","PeriodicalId":94096,"journal":{"name":"Journal of neural engineering","volume":" ","pages":""},"PeriodicalIF":3.8000,"publicationDate":"2025-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12223546/pdf/","citationCount":"0","resultStr":"{\"title\":\"Annotating neurophysiologic data at scale with optimized human input.\",\"authors\":\"Zhongchuan Xu, Brittany H Scheid, Erin C Conrad, Kathryn A Davis, Taneeta Ganguly, Michael A Gelfand, James J Gugger, Xiangyu Jiang, Joshua J LaRocque, William K S Ojemann, Saurabh R Sinha, Genna J Waldman, Joost Wagenaar, Nishant Sinha, Brian Litt\",\"doi\":\"10.1088/1741-2552/ade402\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p><i>Objective.</i>Neuroscience experiments and devices are generating unprecedented volumes of data, but analyzing and validating them presents practical challenges, particularly in annotation. While expert annotation remains the gold standard, it is time consuming to obtain and often poorly reproducible. Although automated annotation approaches exist, they rely on labeled data first to train machine learning algorithms, which limits their scalability. A semi-automated annotation approach that integrates human expertise while optimizing efficiency at scale is critically needed. To address this, we present Annotation Co-pilot, a human-in-the-loop solution that leverages deep active learning (AL) and self-supervised learning (SSL) to improve intracranial EEG (iEEG) annotation, significantly reducing the amount of human annotations.<i>Approach.</i>We automatically annotated iEEG recordings from 28 humans and 4 dogs with epilepsy implanted with two neurodevices that telemetered data to the cloud for analysis. We processed 1500 h of unlabeled iEEG recordings to train a deep neural network using a SSL method Swapping Assignments between View to generate robust, dataset-specific feature embeddings for the purpose of seizure detection. AL was used to select only the most informative data epochs for expert review. We benchmarked this strategy against standard methods.<i>Main result.</i>Over 80 000 iEEG clips, totaling 1176 h of recordings were analyzed. The algorithm matched the best published seizure detectors on two datasets (NeuroVista and NeuroPace responsive neurostimulation) but required, on average, only 1/6 of the human annotations to achieve similar accuracy (area under the ROC curve of 0.9628 ± 0.015) and demonstrated better consistency than human annotators (Cohen's Kappa of 0.95 ± 0.04).<i>Significance</i>. 'Annotation Co-pilot' demonstrated expert-level performance, robustness, and generalizability across two disparate iEEG datasets while reducing annotation time by an average of 83%. This method holds great promise for accelerating basic and translational research in electrophysiology, and potentially accelerating the pathway to clinical translation for AI-based algorithms and devices.</p>\",\"PeriodicalId\":94096,\"journal\":{\"name\":\"Journal of neural engineering\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":3.8000,\"publicationDate\":\"2025-07-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12223546/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of neural engineering\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1088/1741-2552/ade402\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of neural engineering","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1088/1741-2552/ade402","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

目的:神经科学实验和设备正在产生前所未有的数据量,但分析和验证这些数据提出了实际挑战,特别是在注释方面。虽然专家注释仍然是黄金标准,但获取它非常耗时,而且通常很难重现。尽管存在自动标注方法,但它们首先依赖于标记数据来训练机器学习算法,这限制了它们的可扩展性。目前迫切需要一种半自动化的注释方法,该方法在优化大规模效率的同时集成了人类的专业知识。为了解决这个问题,我们提出了注释副驾驶(Annotation Co-pilot),这是一种人在环(HITL)解决方案,利用深度主动学习和自我监督学习来改进脑电图注释,显著减少了人类注释的数量。方法:我们自动注释了18名癫痫患者和4只癫痫狗的颅内脑电图(iEEG)记录,这些记录植入了两个神经设备,将数据遥测到云端进行分析。我们处理了1500小时未标记的脑电图记录,使用自监督学习方法(SwAV)训练深度神经网络,以生成鲁棒的、特定于数据集的特征嵌入,用于癫痫检测。采用主动学习方法选择信息量最大的数据时代供专家评审。我们将此策略与标准方法进行了基准测试。主要结果:分析了超过80,000个eeg片段,总计1,176小时的记录。该算法在两个数据集(NeuroVista和NeuroPace RNS)上匹配了已发表的最佳癫痫检测器,但平均只需要1/6的人类注释即可达到相似的准确性(AUC为0.9628±0.015),并且显示出比人类注释器(Cohen’s Kappa为0.95±0.04)更好的一致性。“标注副驾驶员”在两个不同的iEEG数据集上展示了专家级的性能、鲁棒性和泛化性,同时平均减少了83%的标注时间。这种方法对加速电生理学的基础和转化研究具有很大的希望,并有可能加速基于人工智能的算法和设备的临床转化途径。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Annotating neurophysiologic data at scale with optimized human input.

Objective.Neuroscience experiments and devices are generating unprecedented volumes of data, but analyzing and validating them presents practical challenges, particularly in annotation. While expert annotation remains the gold standard, it is time consuming to obtain and often poorly reproducible. Although automated annotation approaches exist, they rely on labeled data first to train machine learning algorithms, which limits their scalability. A semi-automated annotation approach that integrates human expertise while optimizing efficiency at scale is critically needed. To address this, we present Annotation Co-pilot, a human-in-the-loop solution that leverages deep active learning (AL) and self-supervised learning (SSL) to improve intracranial EEG (iEEG) annotation, significantly reducing the amount of human annotations.Approach.We automatically annotated iEEG recordings from 28 humans and 4 dogs with epilepsy implanted with two neurodevices that telemetered data to the cloud for analysis. We processed 1500 h of unlabeled iEEG recordings to train a deep neural network using a SSL method Swapping Assignments between View to generate robust, dataset-specific feature embeddings for the purpose of seizure detection. AL was used to select only the most informative data epochs for expert review. We benchmarked this strategy against standard methods.Main result.Over 80 000 iEEG clips, totaling 1176 h of recordings were analyzed. The algorithm matched the best published seizure detectors on two datasets (NeuroVista and NeuroPace responsive neurostimulation) but required, on average, only 1/6 of the human annotations to achieve similar accuracy (area under the ROC curve of 0.9628 ± 0.015) and demonstrated better consistency than human annotators (Cohen's Kappa of 0.95 ± 0.04).Significance. 'Annotation Co-pilot' demonstrated expert-level performance, robustness, and generalizability across two disparate iEEG datasets while reducing annotation time by an average of 83%. This method holds great promise for accelerating basic and translational research in electrophysiology, and potentially accelerating the pathway to clinical translation for AI-based algorithms and devices.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信