Mapping the relation between trials methodology and practice-based evidence in the real world of smaller effects: Generalizability and research recommendations.

Abstract

Objective: To address the generalizability of results from trials (evidence-based practice) to routine practice (practice-based evidence), focusing on smaller therapist effects and differential treatment effects.

Method: We utilized data from a pragmatic trial comparing cognitive behavioral therapy (CBT) and person-centered experiential therapy (PCET) as well as routine outcome data from all patients in the clinical organization in which the trial was embedded. We constructed four datasets starting with the trial assessment data and progressively extended the inclusion criteria for therapists and patients to the point of capturing the whole routine outcome dataset across the clinical organization. We applied multilevel modeling to datasets to address the stated objectives.

Results: In the trial data, non-significant therapist effects became significant as a function of increasing inclusivity in the routine practice datasets, while non-significant treatment effects favoring PCET in the trial at 6 months came to favor CBT in all routine datasets. In all four datasets, shorter treatments favored PCET (≈ 6-8 sessions) and longer treatments favored CBT.

Conclusion: Embedding trials within routine practice that uses the same outcome measures enables direct tests of trial generalizability. Recommendations enhancing transparency in trial reporting are made to aid generalizability of trial results to routine practice.