Variation within and between digital pathology and light microscopy for the diagnosis of histopathology slides: blinded crossover comparison study.

Abstract

Background: Digital pathology refers to the conversion of histopathology slides to digital image files for examination on computer workstations as opposed to conventional microscopes. Prior to adoption, it is important to demonstrate pathologists provide equivalent reports when using digital pathology in comparison to bright-field and immunofluorescent light microscopy, the current standard of care.

Objective: A multicentre comparison of digital pathology with light microscopy for reporting of histopathology slides, measuring variation within and between pathologists on both modalities.

Design: A blinded crossover 2000-case study estimating clinical management concordance (identical diagnoses plus differences not affecting patient management). Each sample was assessed twice by four pathologists (once using light microscopy, once using digital pathology, the order randomly assigned and a 6-week gap between viewings). Random-effects logistic regression models, including crossed random-effects terms for case and pathologist, estimated percentage clinical management concordance. Findings were interpreted with reference to 98.3% concordance (Azam AS, Miligy IM, Kimani PKU, Maqbool H, Hewitt K, Rajpoot NM, Snead DRJ. Diagnostic concordance and discordance in digital pathology: a systematic review and meta-analysis. J Clin Pathol 2021;74:448-55. https://doi.org/10.1136/jclinpath-2020-206764).

Setting: Sixteen consultant pathologists, four for each specialty, from six National Health Service laboratories. Experience ranged from 3 to 35 years. Some were early adopters of digital pathology, but the majority were new to digital pathology.

Interventions: Eight viewings per sample (four pathologists with light microscopy and with digital pathology), culminating in a consensus ground truth, enabling measurement of agreement within and between readers. Samples enrolled reflected routine practice, included cancer screening biopsies, and were enriched for areas of difficulty [e.g. dysplasia (7, 10, 11)]. State-of-the-art digital pathology equipment designed for diagnosis, and holding either Conformité Européene or Food and Drug Administration approval, was used.

Main outcome: Intra-pathologist variation between reports issued on digital pathology and light microscopy, inter-pathologist variation against ground-truth diagnosis using light microscopy and digital pathology.

Secondary outcomes: Pathologist-recorded reporting times, along with their confidence in diagnosis, analysis of eye-tracking evaluating examination techniques, and a qualitative study examining attitudes of pathologists and laboratory staff to digital pathology adoption.

Results: Two thousand and twenty-four cases (608 breast, 607 gastrointestinal, 609 skin, 200 renal) were recruited, with breast and gastrointestinal including screening samples [207 (34%) breast, 250 (41%) gastrointestinal]. Overall, in light microscopy versus digital pathology comparisons, clinical management concordance levels were 99.95% (95% confidence interval 99.91 to 99.97). Similar results were observed within specialties [breast: 99.40% (95% confidence interval 99.06 to 99.62); gastrointestinal 99.96% (95% confidence interval 99.89 to 99.99); skin 99.99% (95% confidence interval 99.92 to 100.0); renal 99.99% (95% confidence interval 99.57 to 100.0)], and within screening cases [98.96% (95% confidence interval 98.42 to 99.32), breast 96.27% (94.63 to 97.43), gastrointestinal 99.93% (95% confidence interval 99.68 to 99.98)]. Reporting time between digital pathology and light microscopy was similar, but pathologists became faster on digital pathology with familiarity. Pathologists recorded high levels of confidence in their diagnosis with light microscopy, significantly higher than digital pathology.

Limitations: Cytology cases and specialty groups outside those tested were not examined. The study used two digital pathology scanning systems. Other systems available on the market were not tested.

Conclusions: Clinical management concordance levels between the two modalities exceed the reference 98.3% in breast, gastrointestinal, skin and renal specialties, and pooled breast and large bowel cancer screening cases. Subgroup analysis of clinically significant differences revealed a range of differences including areas where interobserver variability is known to be high, which were distributed between reads performed with both platforms and without apparent trends to either.

Future work: The use of digital pathology for cytology samples remains an area for further research.

Study registration: This study is registered as ISRCTN14513591.

Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 17/84/07) and is published in full in Health Technology Assessment; Vol. 29, No. 30. See the NIHR Funding and Awards website for further award information.