Adam Bagg , Philipp W. Raess , Deborah Rund , Siddharth Bhattacharyya , Joanna Wiszniewska , Alon Horowitz , Darrin Jengehino , Guang Fan , Michelle Huynh , Abdoulaye Sanogo , Irit Avivi , Ben-Zion Katz
{"title":"用于骨髓抽吸物常规分析的新型人工智能(AI)辅助数字显微镜系统的性能评估。","authors":"Adam Bagg , Philipp W. Raess , Deborah Rund , Siddharth Bhattacharyya , Joanna Wiszniewska , Alon Horowitz , Darrin Jengehino , Guang Fan , Michelle Huynh , Abdoulaye Sanogo , Irit Avivi , Ben-Zion Katz","doi":"10.1016/j.modpat.2024.100542","DOIUrl":null,"url":null,"abstract":"<div><p>Bone marrow aspiration (BMA) smear analysis is essential for diagnosis, treatment, and monitoring of a variety of benign and neoplastic hematological conditions. Currently, this analysis is performed by manual microscopy. We conducted a multicenter study to validate a computational microscopy approach with an artificial intelligence–driven decision support system. A total of 795 BMA specimens (615 Romanowsky-stained and 180 Prussian blue–stained) from patients with neoplastic and other clinical conditions were analyzed, comparing the performance of the Scopio Labs X100 Full Field BMA system (test method) with manual microscopy (reference method). The system provided an average of 1,385 ± 536 (range, 0-3,131) cells per specimen for analysis. An average of 39.98 ± 19.64 fields of view (range, 0-140) per specimen were selected by the system for analysis, of them 87% ± 21% (range, 0%-100%) were accepted by the qualified operators. These regions were included in an average of 17.62 ± 7.24 regions of interest (range, 1-50) per specimen. The efficiency, sensitivity, and specificity for primary and secondary marrow aspirate characteristics (maturation, morphology, and count assessment), as well as overall interuser agreement, were evaluated. The test method showed a high correlation with the reference method for comprehensive BMA evaluation, both on Romanowsky- (90.85% efficiency, 81.61% sensitivity, and 92.88% specificity) and Prussian blue–stained samples (90.0% efficiency, 81.94% sensitivity, and 93.38% specificity). The overall agreement between the test and reference methods for BMA assessment was 91.1%. For repeatability and reproducibility, all standard deviations and coefficients of variation values were below the predefined acceptance criteria both for discrete measurements (coefficient of variation below 20%) and differential measurements (SD below 5%). The high degree of correlation between the digital decision support system and manual microscopy demonstrates the potential of this system to provide a high-quality, accurate digital BMA analysis, expediting expert review and diagnosis of BMA specimens, with practical applications including remote BMA evaluation and possibly new opportunities for the research of normal and neoplastic hematopoiesis.</p></div>","PeriodicalId":18706,"journal":{"name":"Modern Pathology","volume":null,"pages":null},"PeriodicalIF":7.1000,"publicationDate":"2024-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0893395224001224/pdfft?md5=088a31cd67a8dac2fce1558078c9ce0e&pid=1-s2.0-S0893395224001224-main.pdf","citationCount":"0","resultStr":"{\"title\":\"Performance Evaluation of a Novel Artificial Intelligence–Assisted Digital Microscopy System for the Routine Analysis of Bone Marrow Aspirates\",\"authors\":\"Adam Bagg , Philipp W. Raess , Deborah Rund , Siddharth Bhattacharyya , Joanna Wiszniewska , Alon Horowitz , Darrin Jengehino , Guang Fan , Michelle Huynh , Abdoulaye Sanogo , Irit Avivi , Ben-Zion Katz\",\"doi\":\"10.1016/j.modpat.2024.100542\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Bone marrow aspiration (BMA) smear analysis is essential for diagnosis, treatment, and monitoring of a variety of benign and neoplastic hematological conditions. Currently, this analysis is performed by manual microscopy. We conducted a multicenter study to validate a computational microscopy approach with an artificial intelligence–driven decision support system. A total of 795 BMA specimens (615 Romanowsky-stained and 180 Prussian blue–stained) from patients with neoplastic and other clinical conditions were analyzed, comparing the performance of the Scopio Labs X100 Full Field BMA system (test method) with manual microscopy (reference method). The system provided an average of 1,385 ± 536 (range, 0-3,131) cells per specimen for analysis. An average of 39.98 ± 19.64 fields of view (range, 0-140) per specimen were selected by the system for analysis, of them 87% ± 21% (range, 0%-100%) were accepted by the qualified operators. These regions were included in an average of 17.62 ± 7.24 regions of interest (range, 1-50) per specimen. The efficiency, sensitivity, and specificity for primary and secondary marrow aspirate characteristics (maturation, morphology, and count assessment), as well as overall interuser agreement, were evaluated. The test method showed a high correlation with the reference method for comprehensive BMA evaluation, both on Romanowsky- (90.85% efficiency, 81.61% sensitivity, and 92.88% specificity) and Prussian blue–stained samples (90.0% efficiency, 81.94% sensitivity, and 93.38% specificity). The overall agreement between the test and reference methods for BMA assessment was 91.1%. For repeatability and reproducibility, all standard deviations and coefficients of variation values were below the predefined acceptance criteria both for discrete measurements (coefficient of variation below 20%) and differential measurements (SD below 5%). The high degree of correlation between the digital decision support system and manual microscopy demonstrates the potential of this system to provide a high-quality, accurate digital BMA analysis, expediting expert review and diagnosis of BMA specimens, with practical applications including remote BMA evaluation and possibly new opportunities for the research of normal and neoplastic hematopoiesis.</p></div>\",\"PeriodicalId\":18706,\"journal\":{\"name\":\"Modern Pathology\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":7.1000,\"publicationDate\":\"2024-06-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.sciencedirect.com/science/article/pii/S0893395224001224/pdfft?md5=088a31cd67a8dac2fce1558078c9ce0e&pid=1-s2.0-S0893395224001224-main.pdf\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Modern Pathology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0893395224001224\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PATHOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Modern Pathology","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0893395224001224","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PATHOLOGY","Score":null,"Total":0}
Performance Evaluation of a Novel Artificial Intelligence–Assisted Digital Microscopy System for the Routine Analysis of Bone Marrow Aspirates
Bone marrow aspiration (BMA) smear analysis is essential for diagnosis, treatment, and monitoring of a variety of benign and neoplastic hematological conditions. Currently, this analysis is performed by manual microscopy. We conducted a multicenter study to validate a computational microscopy approach with an artificial intelligence–driven decision support system. A total of 795 BMA specimens (615 Romanowsky-stained and 180 Prussian blue–stained) from patients with neoplastic and other clinical conditions were analyzed, comparing the performance of the Scopio Labs X100 Full Field BMA system (test method) with manual microscopy (reference method). The system provided an average of 1,385 ± 536 (range, 0-3,131) cells per specimen for analysis. An average of 39.98 ± 19.64 fields of view (range, 0-140) per specimen were selected by the system for analysis, of them 87% ± 21% (range, 0%-100%) were accepted by the qualified operators. These regions were included in an average of 17.62 ± 7.24 regions of interest (range, 1-50) per specimen. The efficiency, sensitivity, and specificity for primary and secondary marrow aspirate characteristics (maturation, morphology, and count assessment), as well as overall interuser agreement, were evaluated. The test method showed a high correlation with the reference method for comprehensive BMA evaluation, both on Romanowsky- (90.85% efficiency, 81.61% sensitivity, and 92.88% specificity) and Prussian blue–stained samples (90.0% efficiency, 81.94% sensitivity, and 93.38% specificity). The overall agreement between the test and reference methods for BMA assessment was 91.1%. For repeatability and reproducibility, all standard deviations and coefficients of variation values were below the predefined acceptance criteria both for discrete measurements (coefficient of variation below 20%) and differential measurements (SD below 5%). The high degree of correlation between the digital decision support system and manual microscopy demonstrates the potential of this system to provide a high-quality, accurate digital BMA analysis, expediting expert review and diagnosis of BMA specimens, with practical applications including remote BMA evaluation and possibly new opportunities for the research of normal and neoplastic hematopoiesis.
期刊介绍:
Modern Pathology, an international journal under the ownership of The United States & Canadian Academy of Pathology (USCAP), serves as an authoritative platform for publishing top-tier clinical and translational research studies in pathology.
Original manuscripts are the primary focus of Modern Pathology, complemented by impactful editorials, reviews, and practice guidelines covering all facets of precision diagnostics in human pathology. The journal's scope includes advancements in molecular diagnostics and genomic classifications of diseases, breakthroughs in immune-oncology, computational science, applied bioinformatics, and digital pathology.