The need for fully-effective federated analytics of data sources for clinical trials.

Abstract

Introduction: Clinical trials are usually analysed in a single environment allowing for flexible analysis including adjustment or stratification by subgroup: 'one-stage' analysis of individual-level data. Health systems datasets, often distributed across geography and providers, can streamline clinical trials. Data are increasingly accessible in secure data environments (SDEs). Future trial analyses may involve working across multiple SDEs. Row-level data and identifiable data often cannot leave, requiring a 'two-stage approach', where summary data from each SDE are meta-analysed.

Objective: To quantify the potential loss of precision and concomitant increases in required sample sizes, and to make recommendations for trial design and conduct, if clinical trial data are split across silos (e.g. SDEs).

Methods: Simulations used data from clinical trials in breast cancer, tuberculosis and prostate cancer with time-to-event, binary and continuous outcome measures. Silos were mimicked by 1000 random partitions into 2, 4, 10 and 25 equal silos and 4 unequal silos proportionate to the UK nations. Data were analysed as if the data could be pooled ignoring silo, pooled accounting for silo (one-stage) or not pooled (two-stage). Estimates and standard errors were presented graphically.

Results: For all three outcome measure types, standard errors increased while point estimates spread out as more silos were introduced. Small biases occurred for binary and time-to-event outcomes. This did not always appreciably reduce efficiency. However, in one example with time-to-event data and the largest number of silos, a near-doubling of sample size would have been required to pre-emptively offset the loss of efficiency.

Conclusion: Any need to use a two-stage analysis approach has a negative effect compared to doing a one-stage analysis. Technical and data governance solutions to support one-stage analyses are recommended.