使用机器学习促进人体健康化学品评估数据提取的概念验证:一项研究方案。

Michelle Angrish, Kristina A Thayer, Brittany Schulz, Artur Nowak, Amanda Persad, Allison L Phillips, Glenn Rice, Teresa Shannon, A Amina Wilkins, Krista Christensen, Elizabeth G Radke, Andrew Shapiro, Michele M Taylor, Vickie R Walker, Andrew A Rooney, Sean M Watford
{"title":"使用机器学习促进人体健康化学品评估数据提取的概念验证:一项研究方案。","authors":"Michelle Angrish, Kristina A Thayer, Brittany Schulz, Artur Nowak, Amanda Persad, Allison L Phillips, Glenn Rice, Teresa Shannon, A Amina Wilkins, Krista Christensen, Elizabeth G Radke, Andrew Shapiro, Michele M Taylor, Vickie R Walker, Andrew A Rooney, Sean M Watford","doi":"10.1080/2833373x.2024.2421192","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Systematic review (SR) methods are relied upon to develop transparent, unbiased, and standardized human health chemical assessments. The expectation is that these assessments will have discovered and evaluated all of the available information in a trackable, transparent, and reproducible manner inherent to SR principles. The challenge is that chemical assessment development relies on mostly literature-based data using manual approaches that are not scalable. Various SR tools have increased the efficiency of assessment development by implementing semi-automated approaches (human in the loop) for data discovery (literature search and screening) and enhanced data repositories with standardized data collection and curation frameworks. Yet filling these repositories with data extractions has remained a manual process and connecting the various tools together in one interoperable workflow remains challenging.</p><p><strong>Objectives: </strong>The objective of this protocol is to explore incorporation of a semi-automated data extraction tool (Dextr) into a chemical assessment workflow and understand if the new tool improves overall user experience.</p><p><strong>Methods: </strong>The workflow will use template systematic evidence map (SEM) methods developed by the Environmental Protection Agency for the identification of included studies. The methods described focus on the data extraction component of the workflow using a fully manual or a semi-automated (human in the loop) data extraction approach. Both the manual and semi-automated data extractions will occur in Dextr. The new data extraction tool will be evaluated for user experience and whether the data extracted using the automated approach meets or exceeds metrics (precision, recall, and F1 score) for a fully manual data extraction.</p><p><strong>Discussion: </strong>Artificial intelligence (AI) and machine learning (ML) methods have rapidly advanced and show promise in achieving operational efficiencies in chemical assessment workflows by supporting automated or semi-automated SR methods, possibly improving the user experience. Yet incorporating advances into sustainable workflows has remained a challenge. Whether using a tool like Dextr improves operational efficiencies and the user experience remains to be determined.</p>","PeriodicalId":520510,"journal":{"name":"Evidence-based toxicology","volume":"2 1","pages":"2421192"},"PeriodicalIF":0.0000,"publicationDate":"2024-11-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12004489/pdf/","citationCount":"0","resultStr":"{\"title\":\"Proof-of-concept for using machine learning to facilitate data extraction for human health chemical assessments: a study protocol.\",\"authors\":\"Michelle Angrish, Kristina A Thayer, Brittany Schulz, Artur Nowak, Amanda Persad, Allison L Phillips, Glenn Rice, Teresa Shannon, A Amina Wilkins, Krista Christensen, Elizabeth G Radke, Andrew Shapiro, Michele M Taylor, Vickie R Walker, Andrew A Rooney, Sean M Watford\",\"doi\":\"10.1080/2833373x.2024.2421192\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>Systematic review (SR) methods are relied upon to develop transparent, unbiased, and standardized human health chemical assessments. The expectation is that these assessments will have discovered and evaluated all of the available information in a trackable, transparent, and reproducible manner inherent to SR principles. The challenge is that chemical assessment development relies on mostly literature-based data using manual approaches that are not scalable. Various SR tools have increased the efficiency of assessment development by implementing semi-automated approaches (human in the loop) for data discovery (literature search and screening) and enhanced data repositories with standardized data collection and curation frameworks. Yet filling these repositories with data extractions has remained a manual process and connecting the various tools together in one interoperable workflow remains challenging.</p><p><strong>Objectives: </strong>The objective of this protocol is to explore incorporation of a semi-automated data extraction tool (Dextr) into a chemical assessment workflow and understand if the new tool improves overall user experience.</p><p><strong>Methods: </strong>The workflow will use template systematic evidence map (SEM) methods developed by the Environmental Protection Agency for the identification of included studies. The methods described focus on the data extraction component of the workflow using a fully manual or a semi-automated (human in the loop) data extraction approach. Both the manual and semi-automated data extractions will occur in Dextr. The new data extraction tool will be evaluated for user experience and whether the data extracted using the automated approach meets or exceeds metrics (precision, recall, and F1 score) for a fully manual data extraction.</p><p><strong>Discussion: </strong>Artificial intelligence (AI) and machine learning (ML) methods have rapidly advanced and show promise in achieving operational efficiencies in chemical assessment workflows by supporting automated or semi-automated SR methods, possibly improving the user experience. Yet incorporating advances into sustainable workflows has remained a challenge. Whether using a tool like Dextr improves operational efficiencies and the user experience remains to be determined.</p>\",\"PeriodicalId\":520510,\"journal\":{\"name\":\"Evidence-based toxicology\",\"volume\":\"2 1\",\"pages\":\"2421192\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-11-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12004489/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Evidence-based toxicology\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1080/2833373x.2024.2421192\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Evidence-based toxicology","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/2833373x.2024.2421192","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

背景:系统评价(SR)方法是建立透明、公正和标准化的人类健康化学评估的基础。期望这些评估将以SR原则固有的可跟踪、透明和可重复的方式发现并评估所有可用的信息。面临的挑战是,化学评估的发展主要依赖于基于文献的数据,使用不可扩展的手动方法。通过实现数据发现(文献搜索和筛选)的半自动方法(人在循环中)以及使用标准化数据收集和管理框架增强的数据存储库,各种SR工具提高了评估开发的效率。然而,用数据提取填充这些存储库仍然是一个手动过程,并且将各种工具连接到一个可互操作的工作流中仍然具有挑战性。目的:本协议的目的是探索将半自动数据提取工具(Dextr)整合到化学评估工作流程中,并了解新工具是否改善了整体用户体验。方法:工作流程将使用环境保护局开发的模板系统证据图(SEM)方法来识别纳入的研究。所描述的方法侧重于使用完全手动或半自动(循环中的人)数据提取方法的工作流的数据提取组件。手动和半自动的数据提取都将在Dextr中进行。新的数据提取工具将评估用户体验,以及使用自动化方法提取的数据是否满足或超过完全手动数据提取的指标(精度、召回率和F1分数)。讨论:人工智能(AI)和机器学习(ML)方法迅速发展,并通过支持自动化或半自动化的SR方法,在实现化学评估工作流程的操作效率方面显示出希望,可能会改善用户体验。然而,将进步纳入可持续工作流程仍然是一项挑战。使用Dextr这样的工具是否能提高作业效率和用户体验还有待确定。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Proof-of-concept for using machine learning to facilitate data extraction for human health chemical assessments: a study protocol.

Background: Systematic review (SR) methods are relied upon to develop transparent, unbiased, and standardized human health chemical assessments. The expectation is that these assessments will have discovered and evaluated all of the available information in a trackable, transparent, and reproducible manner inherent to SR principles. The challenge is that chemical assessment development relies on mostly literature-based data using manual approaches that are not scalable. Various SR tools have increased the efficiency of assessment development by implementing semi-automated approaches (human in the loop) for data discovery (literature search and screening) and enhanced data repositories with standardized data collection and curation frameworks. Yet filling these repositories with data extractions has remained a manual process and connecting the various tools together in one interoperable workflow remains challenging.

Objectives: The objective of this protocol is to explore incorporation of a semi-automated data extraction tool (Dextr) into a chemical assessment workflow and understand if the new tool improves overall user experience.

Methods: The workflow will use template systematic evidence map (SEM) methods developed by the Environmental Protection Agency for the identification of included studies. The methods described focus on the data extraction component of the workflow using a fully manual or a semi-automated (human in the loop) data extraction approach. Both the manual and semi-automated data extractions will occur in Dextr. The new data extraction tool will be evaluated for user experience and whether the data extracted using the automated approach meets or exceeds metrics (precision, recall, and F1 score) for a fully manual data extraction.

Discussion: Artificial intelligence (AI) and machine learning (ML) methods have rapidly advanced and show promise in achieving operational efficiencies in chemical assessment workflows by supporting automated or semi-automated SR methods, possibly improving the user experience. Yet incorporating advances into sustainable workflows has remained a challenge. Whether using a tool like Dextr improves operational efficiencies and the user experience remains to be determined.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信