External performance of the Spinal Infection Treatment Evaluation (SITE) score and Spinal Instability Spondylodiscitis Score (SISS) in predicting operative intervention for de novo spinal infections.

Abstract

Background context: As the incidence of de novo spinal infections has risen with increasing global medical complexity and intravenous drug use, so has the uncertainty around standard of care and surgical decision making. Non-operative management has increased in popularity albeit with frequent failure rates in up to one-third of patients. Although clinical decision making has largely been guided by clinician experience and institutional preference, two recent scoring systems - the Spinal Instability Spondylodiscitis Score (SISS) and the Spinal Infection Treatment Evaluation score (SITE) provide a promising potential avenue towards evidence-based pathways.

Purpose: The aim of the current study was to compare external performance of the SITE and the SISS score in predicting operative decision making in patients with de novo spinal infections seen at a tertiary urban referral center, using real-world clinical decision making as a comparison. A secondary aim was to elucidate areas with low reliability or floor or ceiling effects as possible targets for score improvement.

Study design/setting: Retrospective external validation study utilizing consecutive cases from an academic tertiary referral center PATIENT SAMPLE: Adult patients undergoing treatment for spondylodiscitis or spinal epidural abscess OUTCOME MEASURES: Using the surgical intervention as the ground truth, the primary outcomes were performance metrics of the SITE and SISS score including receiver operating characteristic curves, specificity, sensitivity, and interrater reliability for both score and classification. Of note, the SITE score increases in severity with lower scores, whereas the SISS score increases in severity with higher scores.

Methods: A panel of three blinded raters scored the clinical data.

Results: Two-hundred thirteen patients were included, of which 62% (144/213) underwent non-operative medical management and 38% (80/213) underwent operative management. Mean SITE numerical scores were lower (more severe) in the operative group (5.63 versus 7.45, p < 0.001). The most frequent categorical group for the SITE score was "severe" in both the operative group (93%, 74/80, mean score 5.63) and the nonoperative group (68%, 90/133, mean score 7.45). The mean SISS score did not differ between operative and nonoperative groups (6.73 versus 6.25, p = 0.2). ICC agreement was "almost perfect" for the SITE score (0.86, 95% CI 0.82 - 0.89) and "substantial" for the SISS score (0.68, 95% CI 0.56 - 0.76). Performance metrics for the SITE score were "good" (AUC 0.743, 95% CI 0.67 - 0.81), and for the SISS score were "poor" (AUC 0.557,n95% CI 0.47 - 0.64). ROC analysis for SITE identified a cutoff score of 6.5 to optimize sensitivity and specificity at 0.692 and 0.700, respectively. If using the established cutoff of 8 for "severe" infection as described in the original scoring system, the sensitivity was 0.813, specificity, 0.504, positive predictive value (PPV) 0.496, and negative predictive value (NPV) 0.817. ROC analysis for SISS similarly proposed a cutoff score of 8.0 which yielded a sensitivity and specificity of 0.350 and 0.797, respectively. If using the established cutoff of 10 for "unstable lesion" as described in the original scoring system, the sensitivity was 0.125, specificity 0.917, PPV 0.476, and NPV 0.635..

Conclusions: This study reports external performance metrics for the SITE and SISS score, demonstrating good performance for SITE and poor performance for SISS in predicting operative intervention with almost perfect SITE and substantial SISS agreement among raters. Ceiling effects may limit clinical utility of the SITE score. Subscales which require raters to determine percent vertebral body involvement or posterolateral involvement performed worse. Future work can focus on further discrimination within the "severe" infection group and improvement of low-performing subscales to improve clinical impact.