Gang Liu, Fei Liu, Xu Mao, Xiaoting Xie, Jingyao Sang, Husai Ma, Haiyun Yang, Hui He
{"title":"区分良性和恶性肺磨玻璃结节的多模态深度学习网络","authors":"Gang Liu, Fei Liu, Xu Mao, Xiaoting Xie, Jingyao Sang, Husai Ma, Haiyun Yang, Hui He","doi":"10.2174/0115734056301741240903072017","DOIUrl":null,"url":null,"abstract":"<p><strong>Objective: </strong>This study aimed to establish a multimodal deep-learning network model to enhance the diagnosis of benign and malignant pulmonary ground glass nodules [GGNs].</p><p><strong>Methods: </strong>Retrospective data on pulmonary GGNs were collected from multiple centers across China, including North, Northeast, Northwest, South, and Southwest China. The data were divided into a training set and a validation set in an 8:2 ratio. In addition, a GGN dataset was also obtained from our hospital database and used as the test set. All patients underwent chest computed tomography [CT], and the final diagnosis of the nodules was based on postoperative pathological reports. The Residual Network [ResNet] was used to extract imaging data, the Word2Vec method for semantic information extraction, and the Self Attention method for combining imaging features and patient data to construct a multimodal classification model. Then, the diagnostic efficiency of the proposed multimodal model was compared with that of existing ResNet and VGG models and radiologists.</p><p><strong>Results: </strong>The multicenter dataset comprised 1020 GGNs, including 265 benign and 755 malignant nodules, and the test dataset comprised 204 GGNs, with 67 benign and 137 malignant nodules. In the validation set, the proposed multimodal model achieved an accuracy of 90.2%, a sensitivity of 96.6%, and a specificity of 75.0%, which surpassed that of the VGG [73.1%, 76.7%, and 66.5%] and ResNet [78.0%, 83.3%, and 65.8%] models in diagnosing benign and malignant nodules. In the test set, the multimodal model accurately diagnosed 125 [91.18%] malignant nodules, outperforming radiologists [80.37% accuracy]. Moreover, the multimodal model correctly identified 54 [accuracy, 80.70%] benign nodules, compared to radiologists' accuracy of 85.47%. The consistency test comparing radiologists' diagnostic results with the multimodal model's results in relation to postoperative pathology showed strong agreement, with the multimodal model demonstrating closer alignment with gold standard pathological findings [Kappa=0.720, P<0.01].</p><p><strong>Conclusion: </strong>The multimodal deep learning network model exhibited promising diagnostic effectiveness in distinguishing benign and malignant GGNs and, therefore, holds potential as a reference tool to assist radiologists in improving the diagnostic accuracy of GGNs, potentially enhancing their work efficiency in clinical settings.</p>","PeriodicalId":54215,"journal":{"name":"Current Medical Imaging Reviews","volume":" ","pages":""},"PeriodicalIF":1.1000,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Multimodal Deep Learning Network for Differentiating Between Benign and Malignant Pulmonary Ground Glass Nodules.\",\"authors\":\"Gang Liu, Fei Liu, Xu Mao, Xiaoting Xie, Jingyao Sang, Husai Ma, Haiyun Yang, Hui He\",\"doi\":\"10.2174/0115734056301741240903072017\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Objective: </strong>This study aimed to establish a multimodal deep-learning network model to enhance the diagnosis of benign and malignant pulmonary ground glass nodules [GGNs].</p><p><strong>Methods: </strong>Retrospective data on pulmonary GGNs were collected from multiple centers across China, including North, Northeast, Northwest, South, and Southwest China. The data were divided into a training set and a validation set in an 8:2 ratio. In addition, a GGN dataset was also obtained from our hospital database and used as the test set. All patients underwent chest computed tomography [CT], and the final diagnosis of the nodules was based on postoperative pathological reports. The Residual Network [ResNet] was used to extract imaging data, the Word2Vec method for semantic information extraction, and the Self Attention method for combining imaging features and patient data to construct a multimodal classification model. Then, the diagnostic efficiency of the proposed multimodal model was compared with that of existing ResNet and VGG models and radiologists.</p><p><strong>Results: </strong>The multicenter dataset comprised 1020 GGNs, including 265 benign and 755 malignant nodules, and the test dataset comprised 204 GGNs, with 67 benign and 137 malignant nodules. In the validation set, the proposed multimodal model achieved an accuracy of 90.2%, a sensitivity of 96.6%, and a specificity of 75.0%, which surpassed that of the VGG [73.1%, 76.7%, and 66.5%] and ResNet [78.0%, 83.3%, and 65.8%] models in diagnosing benign and malignant nodules. In the test set, the multimodal model accurately diagnosed 125 [91.18%] malignant nodules, outperforming radiologists [80.37% accuracy]. Moreover, the multimodal model correctly identified 54 [accuracy, 80.70%] benign nodules, compared to radiologists' accuracy of 85.47%. The consistency test comparing radiologists' diagnostic results with the multimodal model's results in relation to postoperative pathology showed strong agreement, with the multimodal model demonstrating closer alignment with gold standard pathological findings [Kappa=0.720, P<0.01].</p><p><strong>Conclusion: </strong>The multimodal deep learning network model exhibited promising diagnostic effectiveness in distinguishing benign and malignant GGNs and, therefore, holds potential as a reference tool to assist radiologists in improving the diagnostic accuracy of GGNs, potentially enhancing their work efficiency in clinical settings.</p>\",\"PeriodicalId\":54215,\"journal\":{\"name\":\"Current Medical Imaging Reviews\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":1.1000,\"publicationDate\":\"2024-09-10\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Current Medical Imaging Reviews\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.2174/0115734056301741240903072017\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Current Medical Imaging Reviews","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.2174/0115734056301741240903072017","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}
Multimodal Deep Learning Network for Differentiating Between Benign and Malignant Pulmonary Ground Glass Nodules.
Objective: This study aimed to establish a multimodal deep-learning network model to enhance the diagnosis of benign and malignant pulmonary ground glass nodules [GGNs].
Methods: Retrospective data on pulmonary GGNs were collected from multiple centers across China, including North, Northeast, Northwest, South, and Southwest China. The data were divided into a training set and a validation set in an 8:2 ratio. In addition, a GGN dataset was also obtained from our hospital database and used as the test set. All patients underwent chest computed tomography [CT], and the final diagnosis of the nodules was based on postoperative pathological reports. The Residual Network [ResNet] was used to extract imaging data, the Word2Vec method for semantic information extraction, and the Self Attention method for combining imaging features and patient data to construct a multimodal classification model. Then, the diagnostic efficiency of the proposed multimodal model was compared with that of existing ResNet and VGG models and radiologists.
Results: The multicenter dataset comprised 1020 GGNs, including 265 benign and 755 malignant nodules, and the test dataset comprised 204 GGNs, with 67 benign and 137 malignant nodules. In the validation set, the proposed multimodal model achieved an accuracy of 90.2%, a sensitivity of 96.6%, and a specificity of 75.0%, which surpassed that of the VGG [73.1%, 76.7%, and 66.5%] and ResNet [78.0%, 83.3%, and 65.8%] models in diagnosing benign and malignant nodules. In the test set, the multimodal model accurately diagnosed 125 [91.18%] malignant nodules, outperforming radiologists [80.37% accuracy]. Moreover, the multimodal model correctly identified 54 [accuracy, 80.70%] benign nodules, compared to radiologists' accuracy of 85.47%. The consistency test comparing radiologists' diagnostic results with the multimodal model's results in relation to postoperative pathology showed strong agreement, with the multimodal model demonstrating closer alignment with gold standard pathological findings [Kappa=0.720, P<0.01].
Conclusion: The multimodal deep learning network model exhibited promising diagnostic effectiveness in distinguishing benign and malignant GGNs and, therefore, holds potential as a reference tool to assist radiologists in improving the diagnostic accuracy of GGNs, potentially enhancing their work efficiency in clinical settings.
期刊介绍:
Current Medical Imaging Reviews publishes frontier review articles, original research articles, drug clinical trial studies and guest edited thematic issues on all the latest advances on medical imaging dedicated to clinical research. All relevant areas are covered by the journal, including advances in the diagnosis, instrumentation and therapeutic applications related to all modern medical imaging techniques.
The journal is essential reading for all clinicians and researchers involved in medical imaging and diagnosis.