Using Aggregated Proficiency Testing Results to Identify Systematic Error.

Abstract

Background: Proficiency testing (PT) should identify systematic errors, which are likely to recur. The Clinical Laboratory Improvement Amendments of 1988 (CLIA) acceptance limits (ALs) include 3 standard deviations (3SD) and concentration limits. We investigated the ability of PT to detect systematic error, especially as affected by the different AL types.

Methods: We removed any ungradable, duplicate, and irregular scores from CLIA laboratory PT data from 2008 to 2018. We calculated the overall miss rate, unsatisfactory event rate, i.e., score <80 (4 of 5 correct), and event rates for score 100 to 0. We used paired t-tests and the Wilcoxon signed-rank test to compare miss rates and unsatisfactory event rates between short- and long-term PT participants. We used the binomial distribution to estimate the expected event scores under the assumption that all misses were independent (random). We compared observed event scores with their expected values as a ratio.

Results: Forty thousand five hundred ninety-six laboratories produced 15 140 128 event scores for 75 analytes. The distribution of event scores was skewed toward multiple event misses (score 0-60) compared to the predicted distribution. Miss rates and unsatisfactory rates were significantly higher for short-term laboratories. Plotting the log ratio of observed vs expected rates for event scores showed that the degree of systematic effect was substantial. The magnitude of the effect was less for 3SD ALs.

Conclusions: In an event, PT misses are often dependent. All ALs detected systematic error. Expressing systematic error using PT data could help to identify and remediate analytical issues.