Gustavo A Jimenez-Maggiora, Michael C Donohue, Michael S Rafii, Rema Raman, Paul S Aisen
{"title":"Artificial intelligence-enabled safety monitoring in Alzheimer's disease clinical trials.","authors":"Gustavo A Jimenez-Maggiora, Michael C Donohue, Michael S Rafii, Rema Raman, Paul S Aisen","doi":"10.1016/j.tjpad.2024.100002","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Investigators conducting clinical trials have an ethical, scientific, and regulatory obligation to protect the safety of trial participants. Traditionally, safety monitoring includes manual review and coding of adverse event data by expert clinicians.</p><p><strong>Objectives: </strong>Our study explores the use of natural language processing (NLP) and artificial intelligence (AI) methods to streamline and standardize clinician coding of adverse event data in Alzheimer's disease (AD) clinical trials.</p><p><strong>Design: </strong>Our quantitative retrospective study aimed to develop a gold standard AD adverse event data set, evaluate the predictive performance of NLP-based models to classify adverse events, and determine whether automated coding is more efficient, accurate, reliable, and consistent than clinician coding.</p><p><strong>Setting: </strong>Our study was conducted at the University of Southern California's Alzheimer's Therapeutic Research Institute (ATRI). ATRI serves as the clinical and data coordinating center for the Alzheimer's Clinical Trial Consortium (ACTC).</p><p><strong>Participants: </strong>We collected demographic and adverse event data from eight completed clinical trials in participants (n=1920) with symptomatic AD conducted between 2005 and 2020.</p><p><strong>Measurements: </strong>Original expert clinician-confirmed codes were used for all model performance comparisons. F1 score was used as the primary model selection metric. Final classifier performance was evaluated using predictive accuracy. Clinician effort was measured in time to code, review, and confirm coded adverse events.</p><p><strong>Results: </strong>In a sample of 1000 adverse events, AI-based AE coding achieved higher accuracy (∼20% increase in accuracy) and was more cost-effective (∼80% cost reduction) than traditional clinician coding.</p><p><strong>Conclusions: </strong>Our study results demonstrate how approaches that effectively combine AI and human expertise can improve the efficiency and quality of adverse event coding and clinical trial safety monitoring.</p>","PeriodicalId":22711,"journal":{"name":"The Journal of Prevention of Alzheimer's Disease","volume":"12 1","pages":"100002"},"PeriodicalIF":4.3000,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"The Journal of Prevention of Alzheimer's Disease","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1016/j.tjpad.2024.100002","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"BUSINESS","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Investigators conducting clinical trials have an ethical, scientific, and regulatory obligation to protect the safety of trial participants. Traditionally, safety monitoring includes manual review and coding of adverse event data by expert clinicians.
Objectives: Our study explores the use of natural language processing (NLP) and artificial intelligence (AI) methods to streamline and standardize clinician coding of adverse event data in Alzheimer's disease (AD) clinical trials.
Design: Our quantitative retrospective study aimed to develop a gold standard AD adverse event data set, evaluate the predictive performance of NLP-based models to classify adverse events, and determine whether automated coding is more efficient, accurate, reliable, and consistent than clinician coding.
Setting: Our study was conducted at the University of Southern California's Alzheimer's Therapeutic Research Institute (ATRI). ATRI serves as the clinical and data coordinating center for the Alzheimer's Clinical Trial Consortium (ACTC).
Participants: We collected demographic and adverse event data from eight completed clinical trials in participants (n=1920) with symptomatic AD conducted between 2005 and 2020.
Measurements: Original expert clinician-confirmed codes were used for all model performance comparisons. F1 score was used as the primary model selection metric. Final classifier performance was evaluated using predictive accuracy. Clinician effort was measured in time to code, review, and confirm coded adverse events.
Results: In a sample of 1000 adverse events, AI-based AE coding achieved higher accuracy (∼20% increase in accuracy) and was more cost-effective (∼80% cost reduction) than traditional clinician coding.
Conclusions: Our study results demonstrate how approaches that effectively combine AI and human expertise can improve the efficiency and quality of adverse event coding and clinical trial safety monitoring.
期刊介绍：
The JPAD Journal of Prevention of Alzheimer’Disease will publish reviews, original research articles and short reports to improve our knowledge in the field of Alzheimer prevention including: neurosciences, biomarkers, imaging, epidemiology, public health, physical cognitive exercise, nutrition, risk and protective factors, drug development, trials design, and heath economic outcomes.JPAD will publish also the meeting abstracts from Clinical Trial on Alzheimer Disease (CTAD) and will be distributed both in paper and online version worldwide.We hope that JPAD with your contribution will play a role in the development of Alzheimer prevention.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
