Establishment and Validation of a Competency-Based Evaluation Indicator System for On-Site Disinfection Service Providers - Shanghai Municipality, China, 2023.

Abstract

Introduction: Effective on-site disinfection of epidemic foci is a critical role in preventing the transmission of infectious diseases. On-site disinfection service providers (ODSP) are a key force in the emergency response systems. This study assessed ODSP capabilities and developed a standardized evaluation indicator system to aid managers in provider selection and supervision.

Methods: A capability evaluation indicator system for ODSP was developed and applied to 88 providers in Shanghai in 2024. Receiver operating characteristic (ROC) curves were plotted by correlating the capability evaluation scores of ODSP with historical disinfection efficacy to determine the optimal threshold. Kappa consistency analysis evaluated the agreement between the new scoring system and historical performance outcomes.

Results: The scoring rates for management systems, material reserves, personnel capability, and quality control were 93.75%, 90.45%, 75.08%, and 51.97%, respectively. ODSP with a history of failed on-site disinfection efficacy evaluations scored significantly lower in personnel capability and quality control than those with consistently passing evaluations (P<0.01). The area under the ROC curve (AUC) for the assessment indicators was 0.954 [95% confidence interval (CI): 0.898, 1], with an optimal capability score threshold of 70 points. The Kappa coefficient was 0.822.

Conclusion: ODSP should further strengthen its quality control measures. There is a significant gap in capability scores among these providers; therfore a hierarchical certification system should be established. The evaluation indicator system for these providers can serve as a tool to assist in dynamic supervision.