Martha Foltyn-Dumitru, Aditya Rastogi, Jaeyoung Cho, Marianne Schell, Mustafa Ahmed Mahmutoglu, Tobias Kessler, Felix Sahm, Wolfgang Wick, Martin Bendszus, Gianluca Brugnara, Philipp Vollmuth
{"title":"基于放射学的机器学习模型的GPT-4高级数据分析的潜力。","authors":"Martha Foltyn-Dumitru, Aditya Rastogi, Jaeyoung Cho, Marianne Schell, Mustafa Ahmed Mahmutoglu, Tobias Kessler, Felix Sahm, Wolfgang Wick, Martin Bendszus, Gianluca Brugnara, Philipp Vollmuth","doi":"10.1093/noajnl/vdae230","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>This study aimed to explore the potential of the Advanced Data Analytics (ADA) package of GPT-4 to autonomously develop machine learning models (MLMs) for predicting glioma molecular types using radiomics from MRI.</p><p><strong>Methods: </strong>Radiomic features were extracted from preoperative MRI of <i>n</i> = 615 newly diagnosed glioma patients to predict glioma molecular types (IDH-wildtype vs IDH-mutant 1p19q-codeleted vs IDH-mutant 1p19q-non-codeleted) with a multiclass ML approach. Specifically, ADA was used to autonomously develop an ML pipeline and benchmark performance against an established handcrafted model using various MRI normalization methods (N4, Zscore, and WhiteStripe). External validation was performed on 2 public glioma datasets D2 (<i>n</i> = 160) and D3 (<i>n</i> = 410).</p><p><strong>Results: </strong>GPT-4 achieved the highest accuracy of 0.820 (95% CI = 0.819-0.821) on the D3 dataset with N4/WS normalization, significantly outperforming the benchmark model's accuracy of 0.678 (95% CI = 0.677-0.680) (<i>P</i> < .001). Class-wise analysis showed performance variations across different glioma types. In the IDH-wildtype group, GPT-4 had a recall of 0.997 (95% CI = 0.997-0.997), surpassing the benchmark's 0.742 (95% CI = 0.740-0.743). For the IDH-mut 1p/19q-non-codel group, GPT-4's recall was 0.275 (95% CI = 0.272-0.279), lower than the benchmark's 0.426 (95% CI = 0.423-0.430). In the IDH-mut 1p/19q-codel group, GPT-4's recall was 0.199 (95% CI = 0.191-0.206), below the benchmark's 0.730 (95% CI = 0.721-0.738). On the D2 dataset, GPT-4's accuracy was significantly lower (<i>P</i> < .001) than the benchmark's, with N4/WS achieving 0.668 (95% CI = 0.666-0.671) compared with 0.719 (95% CI = 0.717-0.722) (<i>P</i> < .001). Class-wise analysis revealed the same pattern as observed in D3.</p><p><strong>Conclusions: </strong>GPT-4 can autonomously develop radiomics-based MLMs, achieving performance comparable to handcrafted MLMs. However, its poorer class-wise performance due to unbalanced datasets shows limitations in handling complete end-to-end ML pipelines.</p>","PeriodicalId":94157,"journal":{"name":"Neuro-oncology advances","volume":"7 1","pages":"vdae230"},"PeriodicalIF":3.7000,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11707530/pdf/","citationCount":"0","resultStr":"{\"title\":\"The potential of GPT-4 advanced data analysis for radiomics-based machine learning models.\",\"authors\":\"Martha Foltyn-Dumitru, Aditya Rastogi, Jaeyoung Cho, Marianne Schell, Mustafa Ahmed Mahmutoglu, Tobias Kessler, Felix Sahm, Wolfgang Wick, Martin Bendszus, Gianluca Brugnara, Philipp Vollmuth\",\"doi\":\"10.1093/noajnl/vdae230\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>This study aimed to explore the potential of the Advanced Data Analytics (ADA) package of GPT-4 to autonomously develop machine learning models (MLMs) for predicting glioma molecular types using radiomics from MRI.</p><p><strong>Methods: </strong>Radiomic features were extracted from preoperative MRI of <i>n</i> = 615 newly diagnosed glioma patients to predict glioma molecular types (IDH-wildtype vs IDH-mutant 1p19q-codeleted vs IDH-mutant 1p19q-non-codeleted) with a multiclass ML approach. Specifically, ADA was used to autonomously develop an ML pipeline and benchmark performance against an established handcrafted model using various MRI normalization methods (N4, Zscore, and WhiteStripe). External validation was performed on 2 public glioma datasets D2 (<i>n</i> = 160) and D3 (<i>n</i> = 410).</p><p><strong>Results: </strong>GPT-4 achieved the highest accuracy of 0.820 (95% CI = 0.819-0.821) on the D3 dataset with N4/WS normalization, significantly outperforming the benchmark model's accuracy of 0.678 (95% CI = 0.677-0.680) (<i>P</i> < .001). Class-wise analysis showed performance variations across different glioma types. In the IDH-wildtype group, GPT-4 had a recall of 0.997 (95% CI = 0.997-0.997), surpassing the benchmark's 0.742 (95% CI = 0.740-0.743). For the IDH-mut 1p/19q-non-codel group, GPT-4's recall was 0.275 (95% CI = 0.272-0.279), lower than the benchmark's 0.426 (95% CI = 0.423-0.430). In the IDH-mut 1p/19q-codel group, GPT-4's recall was 0.199 (95% CI = 0.191-0.206), below the benchmark's 0.730 (95% CI = 0.721-0.738). On the D2 dataset, GPT-4's accuracy was significantly lower (<i>P</i> < .001) than the benchmark's, with N4/WS achieving 0.668 (95% CI = 0.666-0.671) compared with 0.719 (95% CI = 0.717-0.722) (<i>P</i> < .001). Class-wise analysis revealed the same pattern as observed in D3.</p><p><strong>Conclusions: </strong>GPT-4 can autonomously develop radiomics-based MLMs, achieving performance comparable to handcrafted MLMs. However, its poorer class-wise performance due to unbalanced datasets shows limitations in handling complete end-to-end ML pipelines.</p>\",\"PeriodicalId\":94157,\"journal\":{\"name\":\"Neuro-oncology advances\",\"volume\":\"7 1\",\"pages\":\"vdae230\"},\"PeriodicalIF\":3.7000,\"publicationDate\":\"2024-12-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11707530/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Neuro-oncology advances\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1093/noajnl/vdae230\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2025/1/1 0:00:00\",\"PubModel\":\"eCollection\",\"JCR\":\"Q1\",\"JCRName\":\"CLINICAL NEUROLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neuro-oncology advances","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1093/noajnl/vdae230","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/1/1 0:00:00","PubModel":"eCollection","JCR":"Q1","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}
引用次数: 0
摘要
背景：本研究旨在探索GPT-4的高级数据分析（ADA）包的潜力，以自主开发机器学习模型（MLMs），利用MRI放射组学预测胶质瘤分子类型。方法：从n = 615例新诊断的胶质瘤患者的术前MRI中提取放射学特征，采用多分类ML方法预测胶质瘤分子类型（idh -野生型vs idh -突变型1p19q-编码缺失vs idh -突变型1p19q-非编码缺失）。具体来说，ADA被用于自主开发ML管道，并使用各种MRI归一化方法（N4、Zscore和WhiteStripe）对已建立的手工模型进行性能基准测试。在2个公开的胶质瘤数据集D2 （n = 160）和D3 （n = 410）上进行外部验证。结果：GPT-4在N4/WS归一化的D3数据集上达到了0.820 （95% CI = 0.819-0.821）的最高准确率，显著优于基准模型的0.678 (95% CI = 0.677-0.680) （P P P）。结论：GPT-4可以自主开发基于放射组学的传销，达到与手工传销相当的性能。然而，由于不平衡的数据集，其较差的类智能性能显示出在处理完整的端到端ML管道方面的局限性。
The potential of GPT-4 advanced data analysis for radiomics-based machine learning models.
Background: This study aimed to explore the potential of the Advanced Data Analytics (ADA) package of GPT-4 to autonomously develop machine learning models (MLMs) for predicting glioma molecular types using radiomics from MRI.
Methods: Radiomic features were extracted from preoperative MRI of n = 615 newly diagnosed glioma patients to predict glioma molecular types (IDH-wildtype vs IDH-mutant 1p19q-codeleted vs IDH-mutant 1p19q-non-codeleted) with a multiclass ML approach. Specifically, ADA was used to autonomously develop an ML pipeline and benchmark performance against an established handcrafted model using various MRI normalization methods (N4, Zscore, and WhiteStripe). External validation was performed on 2 public glioma datasets D2 (n = 160) and D3 (n = 410).
Results: GPT-4 achieved the highest accuracy of 0.820 (95% CI = 0.819-0.821) on the D3 dataset with N4/WS normalization, significantly outperforming the benchmark model's accuracy of 0.678 (95% CI = 0.677-0.680) (P < .001). Class-wise analysis showed performance variations across different glioma types. In the IDH-wildtype group, GPT-4 had a recall of 0.997 (95% CI = 0.997-0.997), surpassing the benchmark's 0.742 (95% CI = 0.740-0.743). For the IDH-mut 1p/19q-non-codel group, GPT-4's recall was 0.275 (95% CI = 0.272-0.279), lower than the benchmark's 0.426 (95% CI = 0.423-0.430). In the IDH-mut 1p/19q-codel group, GPT-4's recall was 0.199 (95% CI = 0.191-0.206), below the benchmark's 0.730 (95% CI = 0.721-0.738). On the D2 dataset, GPT-4's accuracy was significantly lower (P < .001) than the benchmark's, with N4/WS achieving 0.668 (95% CI = 0.666-0.671) compared with 0.719 (95% CI = 0.717-0.722) (P < .001). Class-wise analysis revealed the same pattern as observed in D3.
Conclusions: GPT-4 can autonomously develop radiomics-based MLMs, achieving performance comparable to handcrafted MLMs. However, its poorer class-wise performance due to unbalanced datasets shows limitations in handling complete end-to-end ML pipelines.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
