外部通用性的挑战:以组织病理学为参考,基于[68Ga]Ga-PSMA-11 PET的放射组学特征用于原发性前列腺癌特征描述的双中心验证的启示》。

IF 4.5 2区 医学 Q1 ONCOLOGY
Cancers Pub Date : 2024-12-07 DOI:10.3390/cancers16234103
Samuele Ghezzo, Praveen Gurunath Bharathi, Heying Duan, Paola Mapelli, Philipp Sorgo, Guido Alejandro Davidzon, Carolina Bezzi, Benjamin Inbeh Chung, Ana Maria Samanes Gajate, Alan Eih Chih Thong, Tommaso Russo, Giorgio Brembilla, Andreas Markus Loening, Pejman Ghanouni, Anna Grattagliano, Alberto Briganti, Francesco De Cobelli, Geoffrey Sonn, Arturo Chiti, Andrei Iagaru, Farshad Moradi, Maria Picchio
{"title":"外部通用性的挑战:以组织病理学为参考,基于[68Ga]Ga-PSMA-11 PET的放射组学特征用于原发性前列腺癌特征描述的双中心验证的启示》。","authors":"Samuele Ghezzo, Praveen Gurunath Bharathi, Heying Duan, Paola Mapelli, Philipp Sorgo, Guido Alejandro Davidzon, Carolina Bezzi, Benjamin Inbeh Chung, Ana Maria Samanes Gajate, Alan Eih Chih Thong, Tommaso Russo, Giorgio Brembilla, Andreas Markus Loening, Pejman Ghanouni, Anna Grattagliano, Alberto Briganti, Francesco De Cobelli, Geoffrey Sonn, Arturo Chiti, Andrei Iagaru, Farshad Moradi, Maria Picchio","doi":"10.3390/cancers16234103","DOIUrl":null,"url":null,"abstract":"<p><p><b>Background</b>: PSMA PET radiomics is a promising tool for primary prostate cancer (PCa) characterisation. However, small single-centre studies and lack of external validation hinder definitive conclusions on the potential of PSMA PET radiomics in the initial workup of PCa. We aimed to validate a radiomics signature in a larger internal cohort and in an external cohort from a separate centre. <b>Methods</b>: One hundred and twenty-seven PCa patients were retrospectively enrolled across two independent hospitals. The first centre (IRCCS San Raffaele Scientific Institute, Centre 1) contributed 62 [<sup>68</sup>Ga]Ga-PSMA-11 PET scans, 20 patients classified as low-grade (ISUP grade < 4), and 42 as high-grade (ISUP grade ≥ 4). The second centre (Stanford University Hospital, Centre 2) provided 65 [<sup>68</sup>Ga]Ga-PSMA-11 PET scans, and 49 low-grade and 16 high-grade patients. A radiomics model previously generated in Centre 1 was tested on the two cohorts separately and afterward on the entire dataset. Then, we evaluated whether the radiomics features selected in the previous investigation could generalise to new data. Several machine learning (ML) models underwent training and testing using 100-fold Monte Carlo cross-validation, independently at both Centre 1 and Centre 2, with a 70-30% train-test split. Additionally, models were trained in one centre and tested in the other, and vice versa. Furthermore, data from both centres were combined for training and testing using Monte Carlo cross-validation. Finally, a new radiomics signature built on this bicentric dataset was proposed. Several performance metrics were computed. <b>Results:</b> The previously generated radiomics signature resulted in an area under the receiver operating characteristic curve (AUC) of 80.4% when tested on Centre 1, while it generalised poorly to Centre 2, where it reached an AUC of 62.7%. When the whole cohort was considered, AUC was 72.5%. Similarly, new ML models trained on the previously selected features yielded, at best, an AUC of 80.9% for Centre 1 and performed at chance for Centre 2 (AUC of 49.3%). A new signature built on this bicentric dataset reached, at best, an average AUC of 91.4% in the test set. <b>Conclusions:</b> The satisfying performance of radiomics models when used in the original development settings, paired with the poor performance otherwise observed, emphasises the need to consider centre-specific factors and dataset characteristics when developing radiomics models. Combining radiomics datasets is a viable strategy to reduce such centre-specific biases, but external validation is still needed.</p>","PeriodicalId":9681,"journal":{"name":"Cancers","volume":"16 23","pages":""},"PeriodicalIF":4.5000,"publicationDate":"2024-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"The Challenge of External Generalisability: Insights from the Bicentric Validation of a [<sup>68</sup>Ga]Ga-PSMA-11 PET Based Radiomics Signature for Primary Prostate Cancer Characterisation Using Histopathology as Reference.\",\"authors\":\"Samuele Ghezzo, Praveen Gurunath Bharathi, Heying Duan, Paola Mapelli, Philipp Sorgo, Guido Alejandro Davidzon, Carolina Bezzi, Benjamin Inbeh Chung, Ana Maria Samanes Gajate, Alan Eih Chih Thong, Tommaso Russo, Giorgio Brembilla, Andreas Markus Loening, Pejman Ghanouni, Anna Grattagliano, Alberto Briganti, Francesco De Cobelli, Geoffrey Sonn, Arturo Chiti, Andrei Iagaru, Farshad Moradi, Maria Picchio\",\"doi\":\"10.3390/cancers16234103\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p><b>Background</b>: PSMA PET radiomics is a promising tool for primary prostate cancer (PCa) characterisation. However, small single-centre studies and lack of external validation hinder definitive conclusions on the potential of PSMA PET radiomics in the initial workup of PCa. We aimed to validate a radiomics signature in a larger internal cohort and in an external cohort from a separate centre. <b>Methods</b>: One hundred and twenty-seven PCa patients were retrospectively enrolled across two independent hospitals. The first centre (IRCCS San Raffaele Scientific Institute, Centre 1) contributed 62 [<sup>68</sup>Ga]Ga-PSMA-11 PET scans, 20 patients classified as low-grade (ISUP grade < 4), and 42 as high-grade (ISUP grade ≥ 4). The second centre (Stanford University Hospital, Centre 2) provided 65 [<sup>68</sup>Ga]Ga-PSMA-11 PET scans, and 49 low-grade and 16 high-grade patients. A radiomics model previously generated in Centre 1 was tested on the two cohorts separately and afterward on the entire dataset. Then, we evaluated whether the radiomics features selected in the previous investigation could generalise to new data. Several machine learning (ML) models underwent training and testing using 100-fold Monte Carlo cross-validation, independently at both Centre 1 and Centre 2, with a 70-30% train-test split. Additionally, models were trained in one centre and tested in the other, and vice versa. Furthermore, data from both centres were combined for training and testing using Monte Carlo cross-validation. Finally, a new radiomics signature built on this bicentric dataset was proposed. Several performance metrics were computed. <b>Results:</b> The previously generated radiomics signature resulted in an area under the receiver operating characteristic curve (AUC) of 80.4% when tested on Centre 1, while it generalised poorly to Centre 2, where it reached an AUC of 62.7%. When the whole cohort was considered, AUC was 72.5%. Similarly, new ML models trained on the previously selected features yielded, at best, an AUC of 80.9% for Centre 1 and performed at chance for Centre 2 (AUC of 49.3%). A new signature built on this bicentric dataset reached, at best, an average AUC of 91.4% in the test set. <b>Conclusions:</b> The satisfying performance of radiomics models when used in the original development settings, paired with the poor performance otherwise observed, emphasises the need to consider centre-specific factors and dataset characteristics when developing radiomics models. Combining radiomics datasets is a viable strategy to reduce such centre-specific biases, but external validation is still needed.</p>\",\"PeriodicalId\":9681,\"journal\":{\"name\":\"Cancers\",\"volume\":\"16 23\",\"pages\":\"\"},\"PeriodicalIF\":4.5000,\"publicationDate\":\"2024-12-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Cancers\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.3390/cancers16234103\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ONCOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cancers","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.3390/cancers16234103","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ONCOLOGY","Score":null,"Total":0}
引用次数: 0

摘要

本文章由计算机程序翻译,如有差异,请以英文原文为准。
The Challenge of External Generalisability: Insights from the Bicentric Validation of a [68Ga]Ga-PSMA-11 PET Based Radiomics Signature for Primary Prostate Cancer Characterisation Using Histopathology as Reference.

Background: PSMA PET radiomics is a promising tool for primary prostate cancer (PCa) characterisation. However, small single-centre studies and lack of external validation hinder definitive conclusions on the potential of PSMA PET radiomics in the initial workup of PCa. We aimed to validate a radiomics signature in a larger internal cohort and in an external cohort from a separate centre. Methods: One hundred and twenty-seven PCa patients were retrospectively enrolled across two independent hospitals. The first centre (IRCCS San Raffaele Scientific Institute, Centre 1) contributed 62 [68Ga]Ga-PSMA-11 PET scans, 20 patients classified as low-grade (ISUP grade < 4), and 42 as high-grade (ISUP grade ≥ 4). The second centre (Stanford University Hospital, Centre 2) provided 65 [68Ga]Ga-PSMA-11 PET scans, and 49 low-grade and 16 high-grade patients. A radiomics model previously generated in Centre 1 was tested on the two cohorts separately and afterward on the entire dataset. Then, we evaluated whether the radiomics features selected in the previous investigation could generalise to new data. Several machine learning (ML) models underwent training and testing using 100-fold Monte Carlo cross-validation, independently at both Centre 1 and Centre 2, with a 70-30% train-test split. Additionally, models were trained in one centre and tested in the other, and vice versa. Furthermore, data from both centres were combined for training and testing using Monte Carlo cross-validation. Finally, a new radiomics signature built on this bicentric dataset was proposed. Several performance metrics were computed. Results: The previously generated radiomics signature resulted in an area under the receiver operating characteristic curve (AUC) of 80.4% when tested on Centre 1, while it generalised poorly to Centre 2, where it reached an AUC of 62.7%. When the whole cohort was considered, AUC was 72.5%. Similarly, new ML models trained on the previously selected features yielded, at best, an AUC of 80.9% for Centre 1 and performed at chance for Centre 2 (AUC of 49.3%). A new signature built on this bicentric dataset reached, at best, an average AUC of 91.4% in the test set. Conclusions: The satisfying performance of radiomics models when used in the original development settings, paired with the poor performance otherwise observed, emphasises the need to consider centre-specific factors and dataset characteristics when developing radiomics models. Combining radiomics datasets is a viable strategy to reduce such centre-specific biases, but external validation is still needed.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Cancers
Cancers Medicine-Oncology
CiteScore
8.00
自引率
9.60%
发文量
5371
审稿时长
18.07 days
期刊介绍: Cancers (ISSN 2072-6694) is an international, peer-reviewed open access journal on oncology. It publishes reviews, regular research papers and short communications. Our aim is to encourage scientists to publish their experimental and theoretical results in as much detail as possible. There is no restriction on the length of the papers. The full experimental details must be provided so that the results can be reproduced.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信