Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergencies

Claudio Xompero, Lorenzo Rossi, Francesca Amoroso, Antonio Bechara Ghobril, Diana Elena Ionita, Eric H. Souied, Carl-Joe Mehanna
{"title":"Pilot study of ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage): a triage tool for ophthalmic emergencies","authors":"Claudio Xompero,&nbsp;Lorenzo Rossi,&nbsp;Francesca Amoroso,&nbsp;Antonio Bechara Ghobril,&nbsp;Diana Elena Ionita,&nbsp;Eric H. Souied,&nbsp;Carl-Joe Mehanna","doi":"10.1016/j.ajoint.2025.100159","DOIUrl":null,"url":null,"abstract":"<div><h3>Introduction</h3><div>ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage) is a GPT-4-based triage tool designed to assess ophthalmic emergencies using a three-tier color-coded system. This study compares ASSORT to the Rescue triage method, using the ophthalmologist’s final assessment as the reference standard.</div></div><div><h3>Materials and methods</h3><div>A prospective study was conducted at the Créteil University Hospital from April to June 2024. Each patient underwent triage using ASSORT, followed by the Rescue triage method. Both tools used the same color-coding system to stratify severity: yellow for emergency cases, green for urgent cases, and white for non-urgent cases. An examining ophthalmologist in their final year of residency performed the final assessment. Concordance between the ophthalmologist and each of the tools was analyzed using Cohen’s kappa coefficient, alongside precision and recall metrics.</div></div><div><h3>Results</h3><div>Fifty-one patients were included. Case severities were distributed as follows: 22/51 white, 27/51 green, and 2/51 yellow, with conjunctivitis (17.5 %) and corneal abrasions (12.5 %) being the two most common presentations. ASSORT demonstrated moderate agreement with the ophthalmologist (κ = 0.54), whereas Rescue showed stronger concordance (κ = 0.85). ASSORT tended to overestimate urgency, assigning more yellow codes than the ophthalmologist. McNemar’s test confirmed significant misclassification by ASSORT (<em>p</em> = 0.0156), while Rescue showed no significant deviation (<em>p</em> = 0.5).</div></div><div><h3>Conclusion</h3><div>While the small sample size limits generalizability, ASSORT shows potential for AI-driven ophthalmic triage but currently overestimates severity compared to the ophthalmologist. Further refinements such as reinforcement learning and multimodal input, as well as large-scale validation are needed to improve accuracy and reduce unnecessary emergency classifications before clinical implementation.</div></div>","PeriodicalId":100071,"journal":{"name":"AJO International","volume":"2 3","pages":"Article 100159"},"PeriodicalIF":0.0000,"publicationDate":"2025-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"AJO International","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2950253525000620","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

Introduction

ASSORT (AI-based Symptom Stratification in Ophthalmology for Rapid Triage) is a GPT-4-based triage tool designed to assess ophthalmic emergencies using a three-tier color-coded system. This study compares ASSORT to the Rescue triage method, using the ophthalmologist’s final assessment as the reference standard.

Materials and methods

A prospective study was conducted at the Créteil University Hospital from April to June 2024. Each patient underwent triage using ASSORT, followed by the Rescue triage method. Both tools used the same color-coding system to stratify severity: yellow for emergency cases, green for urgent cases, and white for non-urgent cases. An examining ophthalmologist in their final year of residency performed the final assessment. Concordance between the ophthalmologist and each of the tools was analyzed using Cohen’s kappa coefficient, alongside precision and recall metrics.

Results

Fifty-one patients were included. Case severities were distributed as follows: 22/51 white, 27/51 green, and 2/51 yellow, with conjunctivitis (17.5 %) and corneal abrasions (12.5 %) being the two most common presentations. ASSORT demonstrated moderate agreement with the ophthalmologist (κ = 0.54), whereas Rescue showed stronger concordance (κ = 0.85). ASSORT tended to overestimate urgency, assigning more yellow codes than the ophthalmologist. McNemar’s test confirmed significant misclassification by ASSORT (p = 0.0156), while Rescue showed no significant deviation (p = 0.5).

Conclusion

While the small sample size limits generalizability, ASSORT shows potential for AI-driven ophthalmic triage but currently overestimates severity compared to the ophthalmologist. Further refinements such as reinforcement learning and multimodal input, as well as large-scale validation are needed to improve accuracy and reduce unnecessary emergency classifications before clinical implementation.
ASSORT(基于人工智能的眼科症状分层快速分类)的试点研究:眼科紧急情况的分类工具
assort(基于人工智能的眼科症状分层快速分类)是一种基于gpt -4的分类工具,旨在使用三层颜色编码系统评估眼科紧急情况。本研究将ASSORT与Rescue分诊法进行比较,以眼科医生的最终评估作为参考标准。材料与方法前瞻性研究于2024年4 - 6月在克柳青大学附属医院进行。每位患者均采用ASSORT分诊法,然后采用Rescue分诊法。这两种工具都使用相同的颜色编码系统来对严重程度进行分层:黄色代表紧急病例,绿色代表紧急病例,白色代表非紧急病例。一名眼科医生在他们实习的最后一年进行了最后的评估。眼科医生和每个工具之间的一致性分析使用科恩的卡帕系数,以及精度和召回指标。结果纳入51例患者。病例严重程度分布如下:22/51白色,27/51绿色和2/51黄色,结膜炎(17.5%)和角膜磨损(12.5%)是两种最常见的表现。ASSORT与眼科医生表现出中等程度的一致性(κ = 0.54),而Rescue表现出较强的一致性(κ = 0.85)。ASSORT倾向于高估紧急程度,比眼科医生分配更多的黄色代码。McNemar 's检验证实ASSORT有显著的误分类(p = 0.0156),而Rescue无显著偏差(p = 0.5)。结论:虽然样本量小限制了普遍性,但与眼科医生相比,ASSORT显示了人工智能驱动的眼科分诊的潜力,但目前高估了严重程度。需要进一步改进,如强化学习和多模式输入,以及大规模验证,以提高准确性并减少临床实施前不必要的紧急分类。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信