Hunjong Lim, Dongjun Choi, Leonard Sunwoo, Jae Hyeop Jung, Sung Hyun Baik, Se Jin Cho, Jinhee Jang, Tackeun Kim, Kyong Joon Lee
{"title":"飞行时间磁共振血管造影自动检测狭窄闭塞病变:观察者表现研究。","authors":"Hunjong Lim, Dongjun Choi, Leonard Sunwoo, Jae Hyeop Jung, Sung Hyun Baik, Se Jin Cho, Jinhee Jang, Tackeun Kim, Kyong Joon Lee","doi":"10.3174/ajnr.A8334","DOIUrl":null,"url":null,"abstract":"<p><strong>Background and purpose: </strong>Intracranial steno-occlusive lesions are responsible for acute ischemic stroke. However, the clinical benefits of artificial intelligence (AI)-based methods for detecting pathologic lesions in intracranial arteries have not been evaluated. We aimed to validate the clinical utility of an AI model for detecting steno-occlusive lesions in the intracranial arteries.</p><p><strong>Materials and methods: </strong>Overall, 138 TOF-MRA images were collected from 2 institutions, which served as internal (<i>n</i> = 62) and external (<i>n</i> = 76) test sets, respectively. Each study was reviewed by 5 radiologists (2 neuroradiologists and 3 radiology residents) to compare the usage and nonusage of our proposed AI model for TOF-MRA interpretation. They identified the steno-occlusive lesions and recorded their reading time. Observer performance was assessed by using the area under the jackknife free-response receiver operating characteristic curve (AUFROC) and reading time for comparison.</p><p><strong>Results: </strong>The average AUFROC for the 5 radiologists demonstrated an improvement from 0.70 without AI to 0.76 with AI (<i>P </i>= .027). Notably, this improvement was most pronounced among the 3 radiology residents, whose performance metrics increased from 0.68 to 0.76 (<i>P </i>= .002). Despite an increased reading time by using AI, there was no significant change among the readings by radiology residents. Moreover, the use of AI resulted in improved interobserver agreement among the reviewers (the intraclass correlation coefficient increased from 0.734 to 0.752).</p><p><strong>Conclusions: </strong>Our proposed AI model offers a supportive tool for radiologists, potentially enhancing the accuracy of detecting intracranial steno-occlusion lesions on TOF-MRA. Less experienced readers may benefit the most from this model.</p>","PeriodicalId":93863,"journal":{"name":"AJNR. American journal of neuroradiology","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11392362/pdf/","citationCount":"0","resultStr":"{\"title\":\"Automated Detection of Steno-Occlusive Lesion on Time-of-Flight MR Angiography: An Observer Performance Study.\",\"authors\":\"Hunjong Lim, Dongjun Choi, Leonard Sunwoo, Jae Hyeop Jung, Sung Hyun Baik, Se Jin Cho, Jinhee Jang, Tackeun Kim, Kyong Joon Lee\",\"doi\":\"10.3174/ajnr.A8334\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background and purpose: </strong>Intracranial steno-occlusive lesions are responsible for acute ischemic stroke. However, the clinical benefits of artificial intelligence (AI)-based methods for detecting pathologic lesions in intracranial arteries have not been evaluated. We aimed to validate the clinical utility of an AI model for detecting steno-occlusive lesions in the intracranial arteries.</p><p><strong>Materials and methods: </strong>Overall, 138 TOF-MRA images were collected from 2 institutions, which served as internal (<i>n</i> = 62) and external (<i>n</i> = 76) test sets, respectively. Each study was reviewed by 5 radiologists (2 neuroradiologists and 3 radiology residents) to compare the usage and nonusage of our proposed AI model for TOF-MRA interpretation. They identified the steno-occlusive lesions and recorded their reading time. Observer performance was assessed by using the area under the jackknife free-response receiver operating characteristic curve (AUFROC) and reading time for comparison.</p><p><strong>Results: </strong>The average AUFROC for the 5 radiologists demonstrated an improvement from 0.70 without AI to 0.76 with AI (<i>P </i>= .027). Notably, this improvement was most pronounced among the 3 radiology residents, whose performance metrics increased from 0.68 to 0.76 (<i>P </i>= .002). Despite an increased reading time by using AI, there was no significant change among the readings by radiology residents. Moreover, the use of AI resulted in improved interobserver agreement among the reviewers (the intraclass correlation coefficient increased from 0.734 to 0.752).</p><p><strong>Conclusions: </strong>Our proposed AI model offers a supportive tool for radiologists, potentially enhancing the accuracy of detecting intracranial steno-occlusion lesions on TOF-MRA. Less experienced readers may benefit the most from this model.</p>\",\"PeriodicalId\":93863,\"journal\":{\"name\":\"AJNR. American journal of neuroradiology\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11392362/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"AJNR. American journal of neuroradiology\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.3174/ajnr.A8334\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"AJNR. American journal of neuroradiology","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3174/ajnr.A8334","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Automated Detection of Steno-Occlusive Lesion on Time-of-Flight MR Angiography: An Observer Performance Study.
Background and purpose: Intracranial steno-occlusive lesions are responsible for acute ischemic stroke. However, the clinical benefits of artificial intelligence (AI)-based methods for detecting pathologic lesions in intracranial arteries have not been evaluated. We aimed to validate the clinical utility of an AI model for detecting steno-occlusive lesions in the intracranial arteries.
Materials and methods: Overall, 138 TOF-MRA images were collected from 2 institutions, which served as internal (n = 62) and external (n = 76) test sets, respectively. Each study was reviewed by 5 radiologists (2 neuroradiologists and 3 radiology residents) to compare the usage and nonusage of our proposed AI model for TOF-MRA interpretation. They identified the steno-occlusive lesions and recorded their reading time. Observer performance was assessed by using the area under the jackknife free-response receiver operating characteristic curve (AUFROC) and reading time for comparison.
Results: The average AUFROC for the 5 radiologists demonstrated an improvement from 0.70 without AI to 0.76 with AI (P = .027). Notably, this improvement was most pronounced among the 3 radiology residents, whose performance metrics increased from 0.68 to 0.76 (P = .002). Despite an increased reading time by using AI, there was no significant change among the readings by radiology residents. Moreover, the use of AI resulted in improved interobserver agreement among the reviewers (the intraclass correlation coefficient increased from 0.734 to 0.752).
Conclusions: Our proposed AI model offers a supportive tool for radiologists, potentially enhancing the accuracy of detecting intracranial steno-occlusion lesions on TOF-MRA. Less experienced readers may benefit the most from this model.