{"title":"利用电子健康记录数据对微创青光眼手术后出血进行辅助分类的 ChatGPT","authors":"","doi":"10.1016/j.xops.2024.100602","DOIUrl":null,"url":null,"abstract":"<div><h3>Purpose</h3><p>To evaluate the performance of a large language model (LLM) in classifying electronic health record (EHR) text, and to use this classification to evaluate the type and resolution of hemorrhagic events (HEs) after microinvasive glaucoma surgery (MIGS).</p></div><div><h3>Design</h3><p>Retrospective cohort study.</p></div><div><h3>Participants</h3><p>Eyes from the Bascom Palmer Glaucoma Repository.</p></div><div><h3>Methods</h3><p>Eyes that underwent MIGS between July 1, 2014 and February 1, 2022 were analyzed. Chat Generative Pre-trained Transformer (ChatGPT) was used to classify deidentified EHR anterior chamber examination text into HE categories (no hyphema, microhyphema, clot, and hyphema). Agreement between classifications by ChatGPT and a glaucoma specialist was evaluated using Cohen’s Kappa and precision-recall (PR) curve. Time to resolution of HEs was assessed using Cox proportional-hazards models. Goniotomy HE resolution was evaluated by degree of angle treatment (90°–179°, 180°–269°, 270°–360°). Logistic regression was used to identify HE risk factors.</p></div><div><h3>Main Outcome Measures</h3><p>Accuracy of ChatGPT HE classification and incidence and resolution of HEs.</p></div><div><h3>Results</h3><p>The study included 434 goniotomy eyes (368 patients) and 528 Schlemm’s canal stent (SCS) eyes (390 patients). Chat Generative Pre-trained Transformer facilitated excellent HE classification (Cohen’s kappa 0.93, area under PR curve 0.968). Using ChatGPT classifications, at postoperative day 1, HEs occurred in 67.8% of goniotomy and 25.2% of SCS eyes (<em>P</em> < 0.001). The 270° to 360° goniotomy group had the highest HE rate (84.0%, <em>P</em> < 0.001). At postoperative week 1, HEs were observed in 43.4% and 11.3% of goniotomy and SCS eyes, respectively (<em>P</em> < 0.001). By postoperative month 1, HE rates were 13.3% and 1.3% among goniotomy and SCS eyes, respectively (<em>P</em> < 0.001). Time to HE resolution differed between the goniotomy angle groups (log-rank <em>P</em> = 0.034); median time to resolution was 10, 10, and 15 days for the 90° to 179°, 180° to 269°, and 270° to 360° groups, respectively. Risk factor analysis demonstrated greater goniotomy angle was the only significant predictor of HEs (odds ratio for 270°–360°: 4.08, <em>P</em> < 0.001).</p></div><div><h3>Conclusions</h3><p>Large language models can be effectively used to classify longitudinal EHR free-text examination data with high accuracy, highlighting a promising direction for future LLM-assisted research and clinical decision support. Hemorrhagic events are relatively common self-resolving complications that occur more often in goniotomy cases and with larger goniotomy treatments. Time to HE resolution differs significantly between goniotomy groups.</p></div><div><h3>Financial Disclosure(s)</h3><p>Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.</p></div>","PeriodicalId":74363,"journal":{"name":"Ophthalmology science","volume":null,"pages":null},"PeriodicalIF":3.2000,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2666914524001386/pdfft?md5=6a7c056392e56a9af8bd6168c9dd77cb&pid=1-s2.0-S2666914524001386-main.pdf","citationCount":"0","resultStr":"{\"title\":\"ChatGPT-Assisted Classification of Postoperative Bleeding Following Microinvasive Glaucoma Surgery Using Electronic Health Record Data\",\"authors\":\"\",\"doi\":\"10.1016/j.xops.2024.100602\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Purpose</h3><p>To evaluate the performance of a large language model (LLM) in classifying electronic health record (EHR) text, and to use this classification to evaluate the type and resolution of hemorrhagic events (HEs) after microinvasive glaucoma surgery (MIGS).</p></div><div><h3>Design</h3><p>Retrospective cohort study.</p></div><div><h3>Participants</h3><p>Eyes from the Bascom Palmer Glaucoma Repository.</p></div><div><h3>Methods</h3><p>Eyes that underwent MIGS between July 1, 2014 and February 1, 2022 were analyzed. Chat Generative Pre-trained Transformer (ChatGPT) was used to classify deidentified EHR anterior chamber examination text into HE categories (no hyphema, microhyphema, clot, and hyphema). Agreement between classifications by ChatGPT and a glaucoma specialist was evaluated using Cohen’s Kappa and precision-recall (PR) curve. Time to resolution of HEs was assessed using Cox proportional-hazards models. Goniotomy HE resolution was evaluated by degree of angle treatment (90°–179°, 180°–269°, 270°–360°). Logistic regression was used to identify HE risk factors.</p></div><div><h3>Main Outcome Measures</h3><p>Accuracy of ChatGPT HE classification and incidence and resolution of HEs.</p></div><div><h3>Results</h3><p>The study included 434 goniotomy eyes (368 patients) and 528 Schlemm’s canal stent (SCS) eyes (390 patients). Chat Generative Pre-trained Transformer facilitated excellent HE classification (Cohen’s kappa 0.93, area under PR curve 0.968). Using ChatGPT classifications, at postoperative day 1, HEs occurred in 67.8% of goniotomy and 25.2% of SCS eyes (<em>P</em> < 0.001). The 270° to 360° goniotomy group had the highest HE rate (84.0%, <em>P</em> < 0.001). At postoperative week 1, HEs were observed in 43.4% and 11.3% of goniotomy and SCS eyes, respectively (<em>P</em> < 0.001). By postoperative month 1, HE rates were 13.3% and 1.3% among goniotomy and SCS eyes, respectively (<em>P</em> < 0.001). Time to HE resolution differed between the goniotomy angle groups (log-rank <em>P</em> = 0.034); median time to resolution was 10, 10, and 15 days for the 90° to 179°, 180° to 269°, and 270° to 360° groups, respectively. Risk factor analysis demonstrated greater goniotomy angle was the only significant predictor of HEs (odds ratio for 270°–360°: 4.08, <em>P</em> < 0.001).</p></div><div><h3>Conclusions</h3><p>Large language models can be effectively used to classify longitudinal EHR free-text examination data with high accuracy, highlighting a promising direction for future LLM-assisted research and clinical decision support. Hemorrhagic events are relatively common self-resolving complications that occur more often in goniotomy cases and with larger goniotomy treatments. Time to HE resolution differs significantly between goniotomy groups.</p></div><div><h3>Financial Disclosure(s)</h3><p>Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.</p></div>\",\"PeriodicalId\":74363,\"journal\":{\"name\":\"Ophthalmology science\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.2000,\"publicationDate\":\"2024-08-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.sciencedirect.com/science/article/pii/S2666914524001386/pdfft?md5=6a7c056392e56a9af8bd6168c9dd77cb&pid=1-s2.0-S2666914524001386-main.pdf\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Ophthalmology science\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2666914524001386\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"OPHTHALMOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ophthalmology science","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2666914524001386","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"OPHTHALMOLOGY","Score":null,"Total":0}
引用次数: 0
摘要
目的 评估大型语言模型(LLM)在对电子健康记录(EHR)文本进行分类时的性能,并利用该分类评估微创青光眼手术(MIGS)后出血事件(HEs)的类型和解决情况。方法 分析2014年7月1日至2022年2月1日期间接受MIGS手术的眼球。使用 Chat Generative Pre-trained Transformer (ChatGPT) 将去标识化的 EHR 前房检查文本分为 HE 类别(无红斑、微红斑、血块和红斑)。ChatGPT 和青光眼专家分类之间的一致性采用 Cohen's Kappa 和精确度-召回 (PR) 曲线进行评估。采用 Cox 比例危险模型评估 HE 的消退时间。巩膜切开术 HE 解决时间按角膜治疗程度(90°-179°、180°-269°、270°-360°)进行评估。主要结果测量ChatGPT HE分类的准确性以及HE的发生率和缓解率。结果该研究包括434只眼球切开术眼睛(368名患者)和528只Schlemm's管支架(SCS)眼睛(390名患者)。Chat Generative Pre-trained Transformer(聊天生成预训练变换器)为出色的 HE 分类提供了便利(Cohen's kappa 0.93,PR 曲线下面积 0.968)。使用 ChatGPT 分类,在术后第 1 天,67.8% 的眼球切开术眼和 25.2% 的 SCS 眼发生了 HE(P < 0.001)。270° 至 360° 角膜切开术组的 HE 发生率最高(84.0%,P < 0.001)。术后第 1 周,分别有 43.4% 和 11.3% 的眼球切开术和 SCS 眼睛出现 HE(P < 0.001)。到术后第 1 个月,开导晶体植入术眼和 SCS 眼的 HE 发生率分别为 13.3% 和 1.3% (P < 0.001)。不同视神经切开角度组的 HE 缓解时间不同(对数秩 P = 0.034);90°至 179°、180°至 269°和 270°至 360°组的中位缓解时间分别为 10 天、10 天和 15 天。结论大语言模型可有效地用于对纵向电子病历自由文本检查数据进行分类,且准确率很高,这为未来的 LLM 辅助研究和临床决策支持指明了方向。出血事件是相对常见的自愈性并发症,更常发生在声门切开术病例中,也更常发生在较大的声门切开术治疗中。不同开腹手术组的 HE 解决时间差异很大。
ChatGPT-Assisted Classification of Postoperative Bleeding Following Microinvasive Glaucoma Surgery Using Electronic Health Record Data
Purpose
To evaluate the performance of a large language model (LLM) in classifying electronic health record (EHR) text, and to use this classification to evaluate the type and resolution of hemorrhagic events (HEs) after microinvasive glaucoma surgery (MIGS).
Design
Retrospective cohort study.
Participants
Eyes from the Bascom Palmer Glaucoma Repository.
Methods
Eyes that underwent MIGS between July 1, 2014 and February 1, 2022 were analyzed. Chat Generative Pre-trained Transformer (ChatGPT) was used to classify deidentified EHR anterior chamber examination text into HE categories (no hyphema, microhyphema, clot, and hyphema). Agreement between classifications by ChatGPT and a glaucoma specialist was evaluated using Cohen’s Kappa and precision-recall (PR) curve. Time to resolution of HEs was assessed using Cox proportional-hazards models. Goniotomy HE resolution was evaluated by degree of angle treatment (90°–179°, 180°–269°, 270°–360°). Logistic regression was used to identify HE risk factors.
Main Outcome Measures
Accuracy of ChatGPT HE classification and incidence and resolution of HEs.
Results
The study included 434 goniotomy eyes (368 patients) and 528 Schlemm’s canal stent (SCS) eyes (390 patients). Chat Generative Pre-trained Transformer facilitated excellent HE classification (Cohen’s kappa 0.93, area under PR curve 0.968). Using ChatGPT classifications, at postoperative day 1, HEs occurred in 67.8% of goniotomy and 25.2% of SCS eyes (P < 0.001). The 270° to 360° goniotomy group had the highest HE rate (84.0%, P < 0.001). At postoperative week 1, HEs were observed in 43.4% and 11.3% of goniotomy and SCS eyes, respectively (P < 0.001). By postoperative month 1, HE rates were 13.3% and 1.3% among goniotomy and SCS eyes, respectively (P < 0.001). Time to HE resolution differed between the goniotomy angle groups (log-rank P = 0.034); median time to resolution was 10, 10, and 15 days for the 90° to 179°, 180° to 269°, and 270° to 360° groups, respectively. Risk factor analysis demonstrated greater goniotomy angle was the only significant predictor of HEs (odds ratio for 270°–360°: 4.08, P < 0.001).
Conclusions
Large language models can be effectively used to classify longitudinal EHR free-text examination data with high accuracy, highlighting a promising direction for future LLM-assisted research and clinical decision support. Hemorrhagic events are relatively common self-resolving complications that occur more often in goniotomy cases and with larger goniotomy treatments. Time to HE resolution differs significantly between goniotomy groups.
Financial Disclosure(s)
Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.