Multicenter Evaluation of Certainty and Reliability in Calciphylaxis Diagnosis Using a Digital Imaging Platform

The robustness of histopathologic criteria for diagnosing calciphylaxis depends both on dermatopathologists’ confidence in the recognition of individual histologic features and on the interrater reliability (IRR) of these assessments. The aim of this study was to quantify interpretive challenges in calciphylaxis diagnosis through the evaluation of generalized and histopathological feature-specific certainty and IRR and establish which features are most reproducibly recognized. An online diagnostic survey was designed to evaluate accuracy, certainty, and IRR. Case materials comprised a clinical vignette, clinical photograph, and whole slide imaging scans of 1 hematoxylin and eosin and von Kossa-stained section in 20 patients with or without calciphylaxis. Dermatopathologists from multiple institutions independently rendered diagnoses and used a checklist to assess the presence of and certainty in specific histopathologic features. Twenty-three board-certified dermatopathologists (62%) representing 16 institutions responded. Diagnostic accuracy (53% vs 80%) and IRR (Krippendorff alpha [KA]) were lower for calciphylaxis cases compared with mimics (0.171 vs 0.257). Necrosis, finely stippled calcium, and intimal fibroplasia of pannicular arterioles most robustly differentiated calciphylaxis from mimics (P < .05). Among histopathologic features, IRR was the highest for ulceration (KA = 0.66) and necrosis (KA = 0.46) and the lowest for perieccrine calcification (KA = 0.19) and intimal fibroplasia of pannicular arterioles (KA = 0.07). Pathologists’ confidence in correctly identifying the presence or absence of each feature correlated linearly with IRR. Digital imaging platforms can facilitate multi-institutional study of diagnostic concordance for uncommon entities. This study identified necrosis, finely stippled calcium, and intimal fibroplasia of pannicular arterioles as the most robust histopathologic features for diagnosing calciphylaxis, although inter-rater concordance for intimal fibroplasia was low. Quantitative trends in pathologists’ subjective assessments of histologic features may help guide the development of diagnostic guidelines and inform medical education.