利用人工智能支持 BRAF 基因突变检测的知情决策。

IF 5.3 2区 医学 Q1 ONCOLOGY
JCO precision oncology Pub Date : 2024-10-01 Epub Date: 2024-10-30 DOI:10.1200/PO.23.00685
Jennifer Webster, Jennifer Ghith, Orion Penner, Christopher H Lieu, Bob J A Schijvenaars
{"title":"利用人工智能支持 BRAF 基因突变检测的知情决策。","authors":"Jennifer Webster, Jennifer Ghith, Orion Penner, Christopher H Lieu, Bob J A Schijvenaars","doi":"10.1200/PO.23.00685","DOIUrl":null,"url":null,"abstract":"<p><strong>Purpose: </strong>Precision oncology relies on accurate and interpretable reporting of testing and mutation rates. Focusing on the <i>BRAFV600</i> mutations in advanced colorectal carcinoma, non-small-cell lung carcinoma, and cutaneous melanoma, we developed a platform displaying testing and mutation rates reported in the literature, which we annotated using an artificial intelligence (AI) and natural language processing (NLP) pipeline.</p><p><strong>Methods: </strong>Using AI, we identified publications that likely reported a testing or mutation rate, filtered publications for cancer type, and identified sentences that likely reported rates. Rates and covariates were subsequently manually curated by three experts. The AI performance was evaluated using precision and recall metrics. We used an interactive platform to explore and present the annotated testing and mutation rates by certain study characteristics.</p><p><strong>Results: </strong>The interactive dashboard, accessible at the BRAF dimensions website, enables users to filter mutation and testing rates with relevant options (eg, country of study, study type, mutation type) and to visualize annotated rates. The AI pipeline demonstrated excellent filtering performance (>90% precision and recall for all target cancer types) and moderate performance for sentence classification (53%-99% precision; ≥75% recall). The manual annotation of testing and mutation rates revealed inter-rater disagreement (testing rate, 19%; mutation rate, 70%), indicating unclear or nonstandard reporting of rates in some publications.</p><p><strong>Conclusion: </strong>Our AI-driven NLP pipeline demonstrated the potential for annotating biomarker testing and mutation rates. The difficulties we encountered highlight the need for more advanced AI-powered literature searching and data extraction, and more consistent reporting of testing rates. These improvements would reduce the risk of misinterpretation or misunderstanding of testing and mutation rates by AI-based technologies and the health care community, with beneficial impacts on clinical decision-making, research, and trial design.</p>","PeriodicalId":14797,"journal":{"name":"JCO precision oncology","volume":null,"pages":null},"PeriodicalIF":5.3000,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Using Artificial Intelligence to Support Informed Decision-Making on <i>BRAF</i> Mutation Testing.\",\"authors\":\"Jennifer Webster, Jennifer Ghith, Orion Penner, Christopher H Lieu, Bob J A Schijvenaars\",\"doi\":\"10.1200/PO.23.00685\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Purpose: </strong>Precision oncology relies on accurate and interpretable reporting of testing and mutation rates. Focusing on the <i>BRAFV600</i> mutations in advanced colorectal carcinoma, non-small-cell lung carcinoma, and cutaneous melanoma, we developed a platform displaying testing and mutation rates reported in the literature, which we annotated using an artificial intelligence (AI) and natural language processing (NLP) pipeline.</p><p><strong>Methods: </strong>Using AI, we identified publications that likely reported a testing or mutation rate, filtered publications for cancer type, and identified sentences that likely reported rates. Rates and covariates were subsequently manually curated by three experts. The AI performance was evaluated using precision and recall metrics. We used an interactive platform to explore and present the annotated testing and mutation rates by certain study characteristics.</p><p><strong>Results: </strong>The interactive dashboard, accessible at the BRAF dimensions website, enables users to filter mutation and testing rates with relevant options (eg, country of study, study type, mutation type) and to visualize annotated rates. The AI pipeline demonstrated excellent filtering performance (>90% precision and recall for all target cancer types) and moderate performance for sentence classification (53%-99% precision; ≥75% recall). The manual annotation of testing and mutation rates revealed inter-rater disagreement (testing rate, 19%; mutation rate, 70%), indicating unclear or nonstandard reporting of rates in some publications.</p><p><strong>Conclusion: </strong>Our AI-driven NLP pipeline demonstrated the potential for annotating biomarker testing and mutation rates. The difficulties we encountered highlight the need for more advanced AI-powered literature searching and data extraction, and more consistent reporting of testing rates. These improvements would reduce the risk of misinterpretation or misunderstanding of testing and mutation rates by AI-based technologies and the health care community, with beneficial impacts on clinical decision-making, research, and trial design.</p>\",\"PeriodicalId\":14797,\"journal\":{\"name\":\"JCO precision oncology\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":5.3000,\"publicationDate\":\"2024-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"JCO precision oncology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1200/PO.23.00685\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/10/30 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q1\",\"JCRName\":\"ONCOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"JCO precision oncology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1200/PO.23.00685","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/10/30 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"ONCOLOGY","Score":null,"Total":0}
引用次数: 0

摘要

目的:精准肿瘤学依赖于准确且可解释的检测和突变率报告。我们以晚期结直肠癌、非小细胞肺癌和皮肤黑色素瘤中的 BRAFV600 突变为重点,开发了一个显示文献中报告的检测率和突变率的平台,并使用人工智能(AI)和自然语言处理(NLP)管道对其进行了注释:利用人工智能,我们确定了可能报告了检测率或突变率的出版物,根据癌症类型过滤了出版物,并确定了可能报告了检测率或突变率的句子。随后,由三位专家对比率和协变量进行人工筛选。人工智能的性能使用精确度和召回率指标进行评估。我们使用了一个交互式平台,按照某些研究特征来探索和展示注释的检测率和突变率:用户可以在 BRAF dimensions 网站上访问交互式仪表板,通过相关选项(如研究国家、研究类型、突变类型)筛选突变率和检测率,并直观显示注释率。人工智能管道显示出卓越的过滤性能(所有目标癌症类型的精确度和召回率均大于 90%)和中等的句子分类性能(精确度为 53%-99%;召回率≥75%)。对测试率和突变率的人工标注显示出评分者之间的意见分歧(测试率为19%;突变率为70%),这表明某些出版物中对比率的报告不明确或不标准:我们的人工智能驱动 NLP 管道展示了注释生物标记物检测率和突变率的潜力。我们遇到的困难突出表明,需要更先进的人工智能驱动的文献搜索和数据提取,以及更一致的检测率报告。这些改进将降低人工智能技术和医疗界对检测和突变率的误读或误解的风险,从而对临床决策、研究和试验设计产生有益的影响。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Using Artificial Intelligence to Support Informed Decision-Making on BRAF Mutation Testing.

Purpose: Precision oncology relies on accurate and interpretable reporting of testing and mutation rates. Focusing on the BRAFV600 mutations in advanced colorectal carcinoma, non-small-cell lung carcinoma, and cutaneous melanoma, we developed a platform displaying testing and mutation rates reported in the literature, which we annotated using an artificial intelligence (AI) and natural language processing (NLP) pipeline.

Methods: Using AI, we identified publications that likely reported a testing or mutation rate, filtered publications for cancer type, and identified sentences that likely reported rates. Rates and covariates were subsequently manually curated by three experts. The AI performance was evaluated using precision and recall metrics. We used an interactive platform to explore and present the annotated testing and mutation rates by certain study characteristics.

Results: The interactive dashboard, accessible at the BRAF dimensions website, enables users to filter mutation and testing rates with relevant options (eg, country of study, study type, mutation type) and to visualize annotated rates. The AI pipeline demonstrated excellent filtering performance (>90% precision and recall for all target cancer types) and moderate performance for sentence classification (53%-99% precision; ≥75% recall). The manual annotation of testing and mutation rates revealed inter-rater disagreement (testing rate, 19%; mutation rate, 70%), indicating unclear or nonstandard reporting of rates in some publications.

Conclusion: Our AI-driven NLP pipeline demonstrated the potential for annotating biomarker testing and mutation rates. The difficulties we encountered highlight the need for more advanced AI-powered literature searching and data extraction, and more consistent reporting of testing rates. These improvements would reduce the risk of misinterpretation or misunderstanding of testing and mutation rates by AI-based technologies and the health care community, with beneficial impacts on clinical decision-making, research, and trial design.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
9.10
自引率
4.30%
发文量
363
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信