Establishing meaningful score differences for the observer-reported communication ability (ORCA) measure using the scale judgment approach with caregivers of individuals with Angelman syndrome.

Abstract

Purpose: The Observer-Reported Communication Ability (ORCA) measure assesses communication ability in individuals living with Angelman syndrome (AS). As gene therapies progress through clinical trials, we need to identify what within-person changes in communication are meaningful to families. This study used the Scale Judgment approach to estimate meaningful score differences (MSDs) on the ORCA measure for AS.

Methods: Caregivers (n = 156) of individuals living with AS completed Scale Judgment tasks showing vignette pairs depicting an individual's communication ability before and after a hypothetical, unspecified treatment. Pairs represented four contexts (lower, higher, expressive, and receptive communication) and score difference magnitudes (2, 4, 6, 8 T-score units). We used an item response theory (IRT) model to estimate a threshold parameter for each vignette pair, then fit a quadratic regression of the parameters on the T-score differences to identify the MSD. ORCA T-scores are calibrated to a mean of 50 and standard deviation of 10 in the AS reference sample.

Results: Across all contexts, the estimated MSD was 1.80 T-score units (95% CI, 1.11-2.50). Estimates were larger for the lower communication range than higher (2.21 vs. 1.37) and similar for expressive (1.57) and receptive (1.58) communication. When stratified by age, genotype, and baseline score, MSDs varied from 0.82 to 2.58, though confidence intervals overlapped across all comparisons. In exploratory analysis of hypothetical treatment caregivers imagined during the task, estimates were largest among caregivers who reported imagining gene therapy (3.41).

Conclusion: Caregivers consistently endorsed small score changes as meaningful, suggesting that small treatment effects carry real significance for families. Accounting for variability across contexts and subgroups, we recommend an MSD of 2.0 to 3.5 T-score points on the ORCA measure for individuals with AS, bounded below by the smallest score difference presented to caregivers during the scale judgment task and above by the largest subgroup estimate.