JCO Clinical Cancer Informatics最新文献

筛选
英文 中文
Artificial Intelligence-Based Digital Histologic Classifier for Prostate Cancer Risk Stratification: Independent Blinded Validation in Patients Treated With Radical Prostatectomy. 基于人工智能的前列腺癌风险分层数字组织学分类器:根治性前列腺切除术患者的独立盲法验证。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-06-01 Epub Date: 2025-06-18 DOI: 10.1200/CCI-24-00292
Magdalena Fay, Ross S Liao, Zaeem M Lone, Chandana A Reddy, Hassan Muhammad, Chensu Xie, Parag Jain, Wei Huang, Hirak S Basu, Sujit S Nair, Dimple Chakravarty, Sean R Williamson, Shilpa Gupta, Christopher Weight, Rajat Roy, George Wilding, Ashutosh K Tewari, Eric A Klein, Omar Y Mian
{"title":"Artificial Intelligence-Based Digital Histologic Classifier for Prostate Cancer Risk Stratification: Independent Blinded Validation in Patients Treated With Radical Prostatectomy.","authors":"Magdalena Fay, Ross S Liao, Zaeem M Lone, Chandana A Reddy, Hassan Muhammad, Chensu Xie, Parag Jain, Wei Huang, Hirak S Basu, Sujit S Nair, Dimple Chakravarty, Sean R Williamson, Shilpa Gupta, Christopher Weight, Rajat Roy, George Wilding, Ashutosh K Tewari, Eric A Klein, Omar Y Mian","doi":"10.1200/CCI-24-00292","DOIUrl":"https://doi.org/10.1200/CCI-24-00292","url":null,"abstract":"<p><strong>Purpose: </strong>Artificial intelligence (AI) tools that identify pathologic features from digitized whole-slide images (WSIs) of prostate cancer (CaP) generate data to predict outcomes. The objective of this study was to evaluate the clinical validity of an AI-enabled prognostic test, PATHOMIQ_PRAD, using a clinical cohort from the Cleveland Clinic.</p><p><strong>Methods: </strong>We conducted a retrospective analysis of PATHOMIQ_PRAD using CaP WSIs from patients who underwent radical prostatectomy (RP) between 2009 and 2022 and did not receive adjuvant therapy. Patients also had Decipher genomic testing available. WSIs were deidentified, anonymized, and outcomes were blinded. Patients were stratified into high-risk and low-risk categories on the basis of predetermined thresholds for PATHOMIQ_PRAD scores (0.45 for biochemical recurrence [BCR] and 0.55 for distant metastasis [DM]).</p><p><strong>Results: </strong>The study included 344 patients who underwent RP with a median follow-up of 4.3 years. Both PathomIQ and Decipher scores were associated with rates of biochemical recurrence-free survival (BCRFS; PathomIQ score >0.45 <i>v</i> ≤0.45, <i>P</i> <.001; Decipher score >0.6 <i>v</i> ≤0.6, <i>P</i> = .002). There were 16 patients who had DM, and 15 were in the high-risk PathomIQ group (Mets Score >0.55). Both PathomIQ and Decipher scores were associated with rates of metastasis-free survival (PathomIQ score >0.55 <i>v</i> ≤0.55, <i>P</i> <.001; Decipher score >0.6 <i>v</i> ≤0.6, <i>P</i> = .0052). Despite the low event rates for metastasis, multivariable regression demonstrated that high PathomIQ score was significantly associated with DM (>0.55 <i>v</i> ≤0.55, hazard ratio, 10.10 [95% CI, 1.28 to 76.92], <i>P</i> = .0284).</p><p><strong>Conclusion: </strong>These findings independently validate PATHOMIQ_PRAD as a reliable predictor of clinical risk in the postprostatectomy setting. PATHOMIQ_PRAD therefore merits prospective evaluation as a risk stratification tool to select patients for adjuvant or early salvage interventions.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2400292"},"PeriodicalIF":3.3,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144327640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Classifying Stereotactic Radiosurgery Patients by Primary Diagnosis Using Natural Language Processing of Clinical Notes. 应用临床记录的自然语言处理对立体定向放射外科患者进行初步诊断分类。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-06-01 Epub Date: 2025-06-13 DOI: 10.1200/CCI-24-00268
Mario Fugal, David Marshall, Alexander V Alekseyenko, Xia Jing, Graham Warren, Jihad Obeid
{"title":"Classifying Stereotactic Radiosurgery Patients by Primary Diagnosis Using Natural Language Processing of Clinical Notes.","authors":"Mario Fugal, David Marshall, Alexander V Alekseyenko, Xia Jing, Graham Warren, Jihad Obeid","doi":"10.1200/CCI-24-00268","DOIUrl":"10.1200/CCI-24-00268","url":null,"abstract":"<p><strong>Purpose: </strong>Accurate identification of the primary tumor diagnosis of patients who have undergone stereotactic radiosurgery (SRS) from electronic health records is a critical but challenging task. Traditional methods of identifying the primary tumor histology relying on International Classification of Diseases (ICD)9 and ICD10 CM codes often fall short in granularity and completeness, particularly for patients with metastatic cancer.</p><p><strong>Methods: </strong>In this study, we propose an approach leveraging natural language processing (NLP) algorithms to enhance the accuracy of extracting primary tumor histology from the patient's electronic records.</p><p><strong>Results: </strong>Through manual annotation of patient data and subsequent algorithm training, we achieved improvements in accuracy and efficiency in primary tumor type classification and finding histology subtypes not available in ICD10 CM.</p><p><strong>Conclusion: </strong>Our findings underscore the value of NLP in refining research processes, identifying patients' cohorts, and improving efficiencies with the goal of potentially improving patient outcomes in SRS treatment.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2400268"},"PeriodicalIF":3.3,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144289743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Enhancing Patient-Trial Matching With Large Language Models: A Scoping Review of Emerging Applications and Approaches. 用大型语言模型增强患者-试验匹配:对新兴应用和方法的范围审查。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-06-01 Epub Date: 2025-06-09 DOI: 10.1200/CCI-25-00071
Hongyu Chen, Xiaohan Li, Xing He, Aokun Chen, James McGill, Emily C Webber, Hua Xu, Mei Liu, Jiang Bian
{"title":"Enhancing Patient-Trial Matching With Large Language Models: A Scoping Review of Emerging Applications and Approaches.","authors":"Hongyu Chen, Xiaohan Li, Xing He, Aokun Chen, James McGill, Emily C Webber, Hua Xu, Mei Liu, Jiang Bian","doi":"10.1200/CCI-25-00071","DOIUrl":"10.1200/CCI-25-00071","url":null,"abstract":"<p><strong>Purpose: </strong>Patient recruitment remains a major bottleneck in clinical trial execution, with inefficient patient-trial matching often causing delays and failures. Recent advancements in large language models (LLMs) offer a promising avenue for automating and improving this process. This scoping review aims to provide a comprehensive synthesis of the emerging applications of LLMs in patient-trial matching.</p><p><strong>Methods: </strong>A comprehensive search was conducted in PubMed, Web of Science, and OpenAlex for literature published between December 1, 2022, and December 31, 2024. Studies were included if they explicitly integrated LLMs into patient-trial matching systems. Data extraction focused on system architectures, patient data processing, eligibility criteria processing, matching techniques, evaluation metrics, and performance.</p><p><strong>Results: </strong>Of the 2,357 studies initially identified, 24 met the inclusion criteria. The majority (21/24) were published in 2024, highlighting the rapid adoption of LLMs in this domain. Most systems used patient-centric matching (17/24), with OpenAI's generative pretrained transformer models being the most commonly used LLM. Core components of these systems included eligibility criteria processing, patient data processing, and matching, with some incorporating retrieval algorithms to enhance computational efficiency. LLM-integrated approaches demonstrated improved accuracy and scalability in patient-trial matching, although challenges such as performance variability, interpretability, and reliance on synthetic data sets remain significant.</p><p><strong>Conclusion: </strong>LLM-based patient-trial matching systems present a transformative opportunity to enhance the efficiency and accuracy of clinical trial recruitment. Despite current limitations related to model generalizability, explainability, and data constraints, future advancements in hybrid modeling strategies, domain-specific fine-tuning, and real-world data set integration could further optimize LLM-based trial matching. Addressing these challenges will be crucial to realizing the full potential of LLMs in streamlining patient recruitment and accelerating clinical trial execution.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2500071"},"PeriodicalIF":3.3,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12169815/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144259398","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Accuracy and Reproducibility of ChatGPT Responses to Breast Cancer Tumor Board Patients. 乳腺癌肿瘤板患者ChatGPT反应的准确性和可重复性。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-06-01 Epub Date: 2025-06-04 DOI: 10.1200/CCI-25-00001
Ning Liao, Cheukfai Li, William J Gradishar, V Suzanne Klimberg, Joshua A Roshal, Taize Yuan, Sanjiv S Agarwala, Vincente K Valero, Sandra M Swain, Julie A Margenthaler, Isabel T Rubio, Sara A Hurvitz, Charles E Geyer, Nancy U Lin, Hope S Rugo, Guochun Zhang, Nanqiu Liu, Charles M Balch
{"title":"Accuracy and Reproducibility of ChatGPT Responses to Breast Cancer Tumor Board Patients.","authors":"Ning Liao, Cheukfai Li, William J Gradishar, V Suzanne Klimberg, Joshua A Roshal, Taize Yuan, Sanjiv S Agarwala, Vincente K Valero, Sandra M Swain, Julie A Margenthaler, Isabel T Rubio, Sara A Hurvitz, Charles E Geyer, Nancy U Lin, Hope S Rugo, Guochun Zhang, Nanqiu Liu, Charles M Balch","doi":"10.1200/CCI-25-00001","DOIUrl":"https://doi.org/10.1200/CCI-25-00001","url":null,"abstract":"<p><strong>Purpose: </strong>We assessed the accuracy and reproducibility of Chat Generative Pre-Trained Transformer's (ChatGPT) recommendations in response to breast cancer patients by comparing generated outputs with consensus expert opinions.</p><p><strong>Methods: </strong>362 consecutive breast cancer patients sourced from a weekly international breast cancer webinar series were submitted to a tumor board of renowned experts. The same 362 clinical patients were also prompted to ChatGPT-4.0 three separate times to examine reproducibility.</p><p><strong>Results: </strong>Only 46% of ChatGPT-generated content was entirely concordant with the recommendations of breast cancer experts, and only 39% of ChatGPT's responses demonstrated inter-response similarity. ChatGPT's responses demonstrated higher concordance with CEN experts in earlier stages of breast cancer (0, I, II, III) compared to advanced (IV) patients (<i>P</i> = .019). There were less accurate responses from ChatGPT when responding to patients involving molecular markers and genetic testing (<i>P</i> = .025), and in patients involving antibody drug conjugates (<i>P</i> = .006). ChatGPT's responses were not necessarily incorrect but often omitted specific details about clinical management. When the same prompt was independently sent to CEN into the model on three occasions, each time by difference users, ChatGPT's responses exhibited variable content and formatting in 68% (246 out of 362) of patients and were entirely consistent with one another in only 32% of responses.</p><p><strong>Conclusion: </strong>Since this promising clinical decision-making support tool is widely used currently by physicians worldwide, it is important for the user to understand its limitations as currently constructed when responding to multidisciplinary breast cancer patients, and for researchers in the field to continue improving its ability with contemporary, accurate and complete breast cancer information. As currently constructed, ChatGPT is not engineered to generate identical outputs to the same input and was less likely to correctly interpret and recommend treatments for complex breast cancer patients.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2500001"},"PeriodicalIF":3.3,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144227518","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Clinical Trial Design Approach to Auditing Language Models in Health Care Setting. 临床试验设计方法审计语言模型在卫生保健设置。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-06-01 Epub Date: 2025-06-03 DOI: 10.1200/CCI-24-00331
Lovedeep Gondara, Jonathan Simkin, Shebnum Devji
{"title":"Clinical Trial Design Approach to Auditing Language Models in Health Care Setting.","authors":"Lovedeep Gondara, Jonathan Simkin, Shebnum Devji","doi":"10.1200/CCI-24-00331","DOIUrl":"https://doi.org/10.1200/CCI-24-00331","url":null,"abstract":"<p><strong>Purpose: </strong>Rapid advancements in natural language processing have led to the development of sophisticated language models. Inspired by their success, these models are now used in health care for tasks such as clinical documentation and medical record classification. However, language models are prone to errors, which can have serious consequences in critical domains such as health care, ensuring that their reliability is essential to maintain patient safety and data integrity.</p><p><strong>Methods: </strong>To address this, we propose an innovative auditing process based on principles from clinical trial design. Our approach involves subject matter experts (SMEs) manually reviewing pathology reports without previous knowledge of the model's classification. This single-blind setup minimizes bias and allows us to apply statistical rigor to assess model performance.</p><p><strong>Results: </strong>Deployed at the British Columbia Cancer Registry, our audit process effectively identified the core issues in the operational models. Early interventions addressed these issues, maintaining data integrity and patient care standards.</p><p><strong>Conclusion: </strong>The audit provides real-world performance metrics and underscores the importance of human-in-the-loop machine learning. Even advanced models require SME oversight to ensure accuracy and reliability. To our knowledge, we have developed the first continuous audit process for language models in health care, modeled after clinical trial principles. This methodology ensures that audits are statistically sound and operationally feasible, setting a new standard for evaluating language models in critical applications.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2400331"},"PeriodicalIF":3.3,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144217490","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Extremity Soft Tissue Sarcoma Reconstruction Nomograms: A Clinicoradiomic, Machine Learning-Powered Predictor of Postoperative Outcomes. 四肢软组织肉瘤重建图:一种临床放射学、机器学习驱动的术后预后预测器。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-06-01 Epub Date: 2025-06-11 DOI: 10.1200/CCI-25-00007
Rami Elmorsi, Luis D Camacho, David D Krijgh, Heather Lyu, Margaret S Roubaud, Keila Torres, Valerae Lewis, Christina L Roland, Alexander F Mericli
{"title":"Extremity Soft Tissue Sarcoma Reconstruction Nomograms: A Clinicoradiomic, Machine Learning-Powered Predictor of Postoperative Outcomes.","authors":"Rami Elmorsi, Luis D Camacho, David D Krijgh, Heather Lyu, Margaret S Roubaud, Keila Torres, Valerae Lewis, Christina L Roland, Alexander F Mericli","doi":"10.1200/CCI-25-00007","DOIUrl":"https://doi.org/10.1200/CCI-25-00007","url":null,"abstract":"<p><strong>Purpose: </strong>The choice of wound closure modality after limb-sparing extremity soft-tissue sarcoma (eSTS) resection is fraught with uncertainty. Leveraging machine learning and clinicoradiomic data, we developed Sarcoma Reconstruction Nomograms (SARCON), a tool that provides probabilistic estimates of five adverse outcomes on the basis of the selected reconstructive modality.</p><p><strong>Methods: </strong>This retrospective cohort study of limb-sparing eSTS resections integrated clinical variables and radiomic features, including eSTS and limb dimensions. Target outcomes included surgical site infections (SSI), wound dehiscence (WD), seroma formation, and minor and major complications. For each outcome, three machine learning classifiers-Logistic Regression with Lasso regularization, Naïve Bayes, and FasterRisk-were developed and evaluated using 10-fold cross-validation (CV), 50 random 80%-20% splits, leave-one-out CV, and a test data set. The best-performing model for each outcome was used to construct a respective nomogram.</p><p><strong>Results: </strong>A total of 316 limb-sparing eSTS resections were analyzed, predominantly located in the thigh (54%), lower leg (17%), and upper arm (11%). Postoperative outcomes included SSI (12%), WD (16%), seroma formation (8.5%), minor complications (34%), and major complications (25%). Logistic Regression with Lasso regularization consistently outperformed the other models across all outcomes, achieving area under the receiver operator curves ranging from 0.83 to 0.93 in all tests.</p><p><strong>Conclusion: </strong>By providing probabilistic estimates of adverse outcomes on the basis of reconstructive modality, SARCON empowers surgeons to anticipate complications and optimize reconstructive strategies.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2500007"},"PeriodicalIF":3.3,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144276574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Prediction of Lymph Node Metastasis in Non-Small Cell Lung Carcinoma Using Primary Tumor Somatic Mutation Data. 利用原发肿瘤体细胞突变数据预测非小细胞肺癌淋巴结转移。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-06-01 Epub Date: 2025-05-30 DOI: 10.1200/CCI-24-00303
Victor Lee, Nicholas S Moore, Joshua Doyle, Daniel Hicks, Patrick Oh, Shari Bodofsky, Sajid Hossain, Abhijit A Patel, Sanjay Aneja, Robert Homer, Henry S Park
{"title":"Prediction of Lymph Node Metastasis in Non-Small Cell Lung Carcinoma Using Primary Tumor Somatic Mutation Data.","authors":"Victor Lee, Nicholas S Moore, Joshua Doyle, Daniel Hicks, Patrick Oh, Shari Bodofsky, Sajid Hossain, Abhijit A Patel, Sanjay Aneja, Robert Homer, Henry S Park","doi":"10.1200/CCI-24-00303","DOIUrl":"https://doi.org/10.1200/CCI-24-00303","url":null,"abstract":"<p><strong>Purpose: </strong>Lymph node metastasis (LNM) significantly affects prognosis and treatment strategies in non-small cell lung cancer (NSCLC). Current diagnostic methods, including imaging and histopathology, have limited sensitivity and specificity. This study aims to develop and evaluate machine learning (ML) models that predict LNM in NSCLC using single-nucleotide polymorphism (SNP) data from The Cancer Genome Atlas.</p><p><strong>Methods: </strong>A cohort of 542 patients with NSCLC with comprehensive SNP data were analyzed. After preprocessing, feature selection was performed using chi-square tests to identify SNPs significantly associated with LNM. Twelve ML models, including Logistic Regression, Naive Bayes, and Support Vector Machines, were trained and evaluated using bootstrapped data sets. Model performance was assessed using metrics such as accuracy, area under the receiver operating characteristic curve (AUC), and F1 score. Shapley additive explanations values were used for feature interpretability, and survival analysis was conducted to assess clinical outcomes.</p><p><strong>Results: </strong>Naive Bayes and Logistic Regression models achieved the highest predictive performance, with median AUCs of 0.93 and 0.91, respectively. Key SNPs, including mutations in <i>TANC2</i>, <i>KCNT2</i>, and <i>CENPF</i>, were consistently identified as predictive features. Survival analysis demonstrated significant differences in outcomes on the basis of model-predicted LNM status (log-rank <i>P</i> = .0268). Feature selection improved model accuracy and robustness, highlighting the biological relevance of selected SNPs.</p><p><strong>Conclusion: </strong>ML models leveraging primary tumor SNP data can enhance LNM prediction in NSCLC, outperforming traditional diagnostic methods. These findings underscore the potential of integrating genomics and ML to develop noninvasive biomarkers, enabling precise risk stratification and personalized treatment strategies in oncology.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2400303"},"PeriodicalIF":3.3,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144188467","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reliability of Large Language Model Knowledge Across Brand and Generic Cancer Drug Names. 大型语言模型知识在品牌和非专利抗癌药物名称中的可靠性。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-06-01 Epub Date: 2025-06-16 DOI: 10.1200/CCI-24-00257
Jack Gallifant, Shan Chen, Sandeep K Jain, Pedro Moreira, Umit Topaloglu, Hugo J W L Aerts, Jeremy L Warner, William G La Cava, Danielle S Bitterman
{"title":"Reliability of Large Language Model Knowledge Across Brand and Generic Cancer Drug Names.","authors":"Jack Gallifant, Shan Chen, Sandeep K Jain, Pedro Moreira, Umit Topaloglu, Hugo J W L Aerts, Jeremy L Warner, William G La Cava, Danielle S Bitterman","doi":"10.1200/CCI-24-00257","DOIUrl":"https://doi.org/10.1200/CCI-24-00257","url":null,"abstract":"<p><strong>Purpose: </strong>To evaluate the performance and consistency of large language models (LLMs) across brand and generic oncology drug names in various clinical tasks, addressing concerns about potential fluctuations in LLM performance because of subtle phrasing differences that could affect patient care.</p><p><strong>Methods: </strong>This study evaluated three LLMs (GPT-3.5-turbo-0125, GPT-4-turbo, and GPT-4o) using drug names from HemOnc ontology. The assessment included 367 generic-to-brand and 2,516 brand-to-generic pairs, 1,000 drug-drug interaction (DDI) synthetic patient cases, and 2,438 immune-related adverse event (irAE) cases. LLMs were tested on drug name recognition, word association, DDI (DDI) detection, and irAE diagnosis using both brand and generic drug names.</p><p><strong>Results: </strong>LLMs demonstrated high accuracy in matching brand and generic names (GPT-4o: 97.38% for brand, 94.71% for generic, <i>P</i> < .01). However, they showed significant inconsistencies in word association tasks. GPT-3.5-turbo-0125 exhibited biases favoring brand names for effectiveness (odds ratio [OR], 1.43, <i>P</i> < .05) and being side-effect-free (OR, 1.76, <i>P</i> < .05). DDI detection accuracy was poor across all models (<26%), with no significant differences between brand and generic names. Sentiment analysis revealed significant differences, particularly in GPT-3.5-turbo-0125 (brand mean 0.67, generic mean 0.95, <i>P</i> < .01). Consistency in irAE diagnosis varied across models.</p><p><strong>Conclusion: </strong>Despite high proficiency in name-matching, LLMs exhibit inconsistencies when processing brand versus generic drug names in more complex tasks. These findings highlight the need for increased awareness, improved robustness assessment methods, and the development of more consistent systems for handling nomenclature variations in clinical applications of LLMs.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2400257"},"PeriodicalIF":3.3,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144310800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Timing and Method of Patient-Provider Communication for Abnormal Hepatocellular Carcinoma Screening Results in Cirrhosis. 肝硬化患者异常肝癌筛查结果的医患沟通时机与方法。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-05-01 Epub Date: 2025-05-05 DOI: 10.1200/CCI-24-00269
Jeremy Louissaint, Beverly Kyalwazi, John Deng, Timothy P Hogan, Robert W Turer, Elliot B Tapper, David E Gerber, Bryan D Steitz, Sarah R Lieber, Amit G Singal
{"title":"Timing and Method of Patient-Provider Communication for Abnormal Hepatocellular Carcinoma Screening Results in Cirrhosis.","authors":"Jeremy Louissaint, Beverly Kyalwazi, John Deng, Timothy P Hogan, Robert W Turer, Elliot B Tapper, David E Gerber, Bryan D Steitz, Sarah R Lieber, Amit G Singal","doi":"10.1200/CCI-24-00269","DOIUrl":"https://doi.org/10.1200/CCI-24-00269","url":null,"abstract":"<p><strong>Purpose: </strong>Patients with cirrhosis undergo frequent abdominal imaging including semiannual hepatocellular carcinoma (HCC) screening, with results released immediately via the patient portal. We characterized time from patient review to patient-provider communication (PPC) for patients with abnormal liver imaging results.</p><p><strong>Methods: </strong>We identified patients with cirrhosis enrolled in the patient portal with a new abnormal liver lesion (LI-RADS, LR) on ambulatory liver ultrasound (US) or multiphasic computed tomography/magnetic resonance imaging. Imaging findings were grouped into low-risk (US-2, LR-2), intermediate-risk (US-3, LR-3), and high-risk (LR-4, LR-5, LR-M, LR-TIV) results. We extracted three date-time events from the electronic health record, including result release to the patient, patient review of the result, and result-related PPC. We compared communication methods and the median time with PPC after patient review of results between groups.</p><p><strong>Results: </strong>The cohort included 133 patients (median age, 62 years, 56% male) with 34 (25.6%) low-risk, 61 (45.9%) intermediate-risk, and 38 (28.6%) high-risk results. PPC for high-risk results was predominantly via telephone calls (60.5%), whereas portal messages were most commonly used for low- and intermediate-risk results (61.8% and 45.9%, respectively; <i>P</i> < .001). For patients who reviewed their result on the portal, most (79.3%) reviewed the result before PPC, among whom the median time between review and PPC was 55.8 (IQR, 22.0-219.0), 167 (IQR, 42.7-324.0), and 47.3 (IQR, 25.8-78.8) hours for low-, intermediate-, and high-risk results, respectively (<i>P</i> = .02).</p><p><strong>Conclusion: </strong>Portal-based review of abnormal imaging results by patients before provider communication is common, including results concerning a new HCC diagnosis. Further studies are needed to evaluate patient-reported outcomes, such as psychological distress, associated with this method of disclosing cancer-related results.</p>","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2400269"},"PeriodicalIF":3.3,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144052306","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Data Events Are Safety Events: High-Reliability Organization Approach to Improving Data Quality and Safety. 数据事件是安全事件:提高数据质量和安全性的高可靠性组织方法。
IF 3.3
JCO Clinical Cancer Informatics Pub Date : 2025-05-01 Epub Date: 2025-04-30 DOI: 10.1200/CCI-24-00273
Kelly Merriman, Emily Yu, Andrea Hawkins-Daarud, Kerin Adelson, Kenna Shaw, Dan Shoenthal, Jay Patel, Janna Baganz, Andy Futreal, David A Jaffray, Jose Rivera, Caroline Chung
{"title":"Data Events Are Safety Events: High-Reliability Organization Approach to Improving Data Quality and Safety.","authors":"Kelly Merriman, Emily Yu, Andrea Hawkins-Daarud, Kerin Adelson, Kenna Shaw, Dan Shoenthal, Jay Patel, Janna Baganz, Andy Futreal, David A Jaffray, Jose Rivera, Caroline Chung","doi":"10.1200/CCI-24-00273","DOIUrl":"https://doi.org/10.1200/CCI-24-00273","url":null,"abstract":"","PeriodicalId":51626,"journal":{"name":"JCO Clinical Cancer Informatics","volume":"9 ","pages":"e2400273"},"PeriodicalIF":3.3,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12058365/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144057168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信