Obtaining Valid Compatibility Intervals for Sequence Symmetry Analyses Utilizing Active Comparators: A Simulation Study.

Purpose: To compare different methods of estimating 95% compatibility intervals (CIs) for the sequence ratio (SR) when performing a sequence symmetry analysis using an active comparator to reduce the risk of time-varying confounding.

Methods: We conducted a simulation study, where we simulated drug-outcome and outcome-drug sequences for a drug of interest and a comparator drug using the binomial distribution and obtained active comparator SRs and 95% CIs. We simulated scenarios with sample sizes between 5 and 50 observed sequences for each SR, which could take values of 0.5, 1.0, or 2.0, yielding 276 scenarios that were replicated 5000 times. For each replication, we calculated 95% CIs using current recommendations based on exact CIs, the Woolf logit, Baptista-Pike mid-p, and Miettinen-Nurminen score estimator and calculated coverage for each scenario.

Results: All interval estimators provided acceptable coverage when sample sizes exceeded 15, except for the current recommendation, the exact Clopper-Pearson interval. The Miettinen-Nurminen score (coverage 0.951) and Baptista-Pike mid-p interval (coverage 0.955) offered more accurate coverage than other methods. The largest divergence from 0.95 was observed for the current recommendations (coverage 0.979).

Conclusions: The Miettinen-Nurminen score estimator provided the most accurate coverage for 95% CIs of active comparator SRs, especially with low sample sizes. Therefore, we recommend using the Miettinen-Nurminen score estimator for active comparator SRs.