Generalizability in real-world trials

Abstract

Real-world evidence (RWE) trials have a key advantage over conventional randomized controlled trials (RCTs) due to their potentially better generalizability. High generalizability of study results facilitates new biological insights and enables targeted therapeutic strategies. Random sampling of RWE trial participants is regarded as the gold standard for generalizability. Additionally, the use of sample correction procedures can increase the generalizability of trial results, even when using nonrandomly sampled real-world data (RWD). This study presents descriptive evidence on the extent to which the design of currently planned or already conducted RWE trials takes sampling into account. It also examines whether random sampling or procedures for correcting nonrandom samples are considered. Based on text mining of publicly available metadata provided during registrations of RWE trials on clinicaltrials.gov, EU-PAS, and the OSF-RWE registry, it is shown that the share of RWE trial registrations with information on sampling increased from 65.27% in 2002 to 97.43% in 2022, with a corresponding increase from 14.79% to 28.30% for trials with random samples. For RWE trials with nonrandom samples, there is an increase from 0.00% to 0.95% of trials in which sample correction procedures are used. We conclude that the potential benefits of RWD in terms of generalizing trial results are not yet being fully realized.