Tony K W Hung, Jun J Mao, Alan L Ho, Eric J Sherman, Mark Robson, Jae Park, Eytan M Stein, Gilad J Kuperman, David G Pfister
{"title":"Real-world evaluation of user engagement with an artificial intelligence-powered clinical trial application in oncology.","authors":"Tony K W Hung, Jun J Mao, Alan L Ho, Eric J Sherman, Mark Robson, Jae Park, Eytan M Stein, Gilad J Kuperman, David G Pfister","doi":"10.1093/jamia/ocaf129","DOIUrl":null,"url":null,"abstract":"<p><strong>Objectives: </strong>This quality improvement study implemented and prospectively examined user engagement with an artificial intelligence (AI)-powered clinical trial knowledge management application at an NCI-designated comprehensive cancer center.</p><p><strong>Materials and methods: </strong>We prospectively auto-captured user engagement measures from July 1, 2022 to February 29, 2024. Measurement included: (1) event: an app interaction; (2) session: group of events within single setting; (3) engaged session: session longer than 10 s; (4) engagement time; (5) app downloads; (6) active user; and (7) stickiness: monthly active users per normalized total downloads. We analyzed the measures using time series and linear regression.</p><p><strong>Results: </strong>During a 20-month evaluation, the application supported 138 clinical trials, recorded 136 632 user interactions, including 2754 engaged sessions with an average engagement time of 6 min 31 s. Of 243 downloads, 228 (94%) users remained active, with an estimated stickiness score of 3.12 (SD 0.91), indicating sustained provider engagement.</p><p><strong>Discussion: </strong>This study provided insights into the feasibility and potential for integrating an AI-powered clinical trial knowledge management application into oncology workflows, with sustained engagement among providers over a 20-month period. High rates of active users and session stickiness suggest that such application offered meaningful utility in real-world clinical settings, underscoring the need for future studies to assess optimal integration strategies and impact on clinical trial accrual.</p><p><strong>Conclusion: </strong>This study addresses an important gap in the literature regarding the real-world integration of AI technologies in oncology care and offers valuable insights for future research and clinical practice.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1562-1569"},"PeriodicalIF":4.6000,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12451935/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of the American Medical Informatics Association","FirstCategoryId":"91","ListUrlMain":"https://doi.org/10.1093/jamia/ocaf129","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
Objectives: This quality improvement study implemented and prospectively examined user engagement with an artificial intelligence (AI)-powered clinical trial knowledge management application at an NCI-designated comprehensive cancer center.
Materials and methods: We prospectively auto-captured user engagement measures from July 1, 2022 to February 29, 2024. Measurement included: (1) event: an app interaction; (2) session: group of events within single setting; (3) engaged session: session longer than 10 s; (4) engagement time; (5) app downloads; (6) active user; and (7) stickiness: monthly active users per normalized total downloads. We analyzed the measures using time series and linear regression.
Results: During a 20-month evaluation, the application supported 138 clinical trials, recorded 136 632 user interactions, including 2754 engaged sessions with an average engagement time of 6 min 31 s. Of 243 downloads, 228 (94%) users remained active, with an estimated stickiness score of 3.12 (SD 0.91), indicating sustained provider engagement.
Discussion: This study provided insights into the feasibility and potential for integrating an AI-powered clinical trial knowledge management application into oncology workflows, with sustained engagement among providers over a 20-month period. High rates of active users and session stickiness suggest that such application offered meaningful utility in real-world clinical settings, underscoring the need for future studies to assess optimal integration strategies and impact on clinical trial accrual.
Conclusion: This study addresses an important gap in the literature regarding the real-world integration of AI technologies in oncology care and offers valuable insights for future research and clinical practice.
期刊介绍：
JAMIA is AMIA''s premier peer-reviewed journal for biomedical and health informatics. Covering the full spectrum of activities in the field, JAMIA includes informatics articles in the areas of clinical care, clinical research, translational science, implementation science, imaging, education, consumer health, public health, and policy. JAMIA''s articles describe innovative informatics research and systems that help to advance biomedical science and to promote health. Case reports, perspectives and reviews also help readers stay connected with the most important informatics developments in implementation, policy and education.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
