Coefficients of Repeatability for Likely Change: Comparison between PROMIS Computer Adaptive Tests and Short Forms.

Abstract

Objectives: The study identified necessary score changes to categorize patients as likely to have changed in PROMIS computer adaptive testing (CAT) and to compare these with selected 4-item or 8-item short forms (SF 4a or 8a).

Methods: In a large health system symptom management intervention project, 5823 completed PROMIS CAT with the stopping rule of standard error < 3 for anxiety (v1.0), depression (v1.0), pain interference (v1.1), and physical function (v2.0) at baseline and at 4 months. Coefficients of repeatability (CRs) for likely change were determined using 68 % confidence level based on the reliable change index statistics using item response theory standard errors, which were compared with those using 90 % confidence level. CRs for 8-item or 4-item short forms (SFs) were estimated for comparison using established item parameters.

Results: CRs at 68 % confidence ranged from |3| to |4| for CAT and SF 4a and |2| to |3| for SF 8a across domains for baseline scores in the mildly-to-severely impaired ranges. For baseline scores within normal limits across improvement or worsening, CRs at 68 % confidence were |3|-|8| for CAT, |3|-|14| for SF 4a, and |2|-|14| for SF 8a. Across score ranges, directions of change, and domains, relaxing confidence level from 90 % to 68 % lowered the change thresholds by up to 7 points for CAT, 14 points for SF 8a, and 14 points for SF 4a, with the largest reductions for baseline scores within the normal range.

Conclusions: In symptomatic score ranges, smaller and somewhat constant CRs were observed compared to scores within normal ranges. SF 8a achieved CR of 1 point lower than CAT with 4 more items on average in the symptomatic score range. CAT achieved lower CR than either SF length in the healthiest score range.