Conceptualizing meaningful between-group difference in change over time: a demonstration of possible viewpoints.

Abstract

Purpose: Determining if group-level differences in health outcomes are meaningful has recently been neglected in favour of determining if individuals have experienced a meaningful change. We explore interpretation of a meaningful between-group difference (MBGD) in clinical outcome assessment scores, primarily in the context of randomized clinical trials.

Methods: We constructed a series of possible 'viewpoints' on how to conceptualize MBGD thresholds. Each viewpoint is discussed critically in terms of potential advantages and disadvantages, with simulated data to facilitate their consideration.

Results: Five viewpoints are presented and discussed. The first considers whether thresholds for meaningful within-individual change over time can be equally applied at the group-level, which is shown to be untenable. Viewpoints 2-4 consider what would have to be observed in treatment groups to conclude a meaningful between-group difference has occurred, framed in terms of the proportion of patients perceiving that they had meaningfully improved. The final viewpoint considers an alternative framework where stakeholders are directly questioned on the meaningfulness of varying magnitudes of between-group differences. The choice of a single threshold versus general interpretative guidelines is discussed.

Conclusion: There does not appear to be a single method with clear face validity for determining MBGD thresholds. Additionally, the notion that such thresholds can be purely data-driven is challenged, where a degree of subjective stakeholder judgement is likely required. Areas for future research are proposed, to move towards robust method development.