Cohort data with dropout: a simulation study comparing five longitudinal analysis methods.

Abstract

Background: A simulation study was performed to visually demonstrate the problems with repeated measures ANOVA (RMA) and t-tests (TT) compared to linear mixed effects (LME), covariance pattern (CP) or generalized estimating equations (GEE) models in longitudinal cohort studies with dropout.

Methods: Data were generated for a realistic, observational study on health-related quality of life (HRQoL) in a small, heterogeneous sample of children undergoing anti-reflux surgery. Each generated sample comprised two groups: one with low levels (4-10%) of random dropout (missing completely at random, MCAR); the other with higher levels (10-40%), where the chance of dropout depended on lower baseline HRQoL (missing at random, MAR). Outcome data were simulated for four time points in a one-year period, assuming in both groups small but meaningful increases in HRQoL between baseline and 3 months, and thereafter constant levels to 12 months. Five analysis methods were applied to the simulated datasets: LME; CP; GEE; RMA; and independent TT at all time points (between groups) or paired TT on the difference between 12 and 0 months (within groups). The bias in estimated marginal means was examined, and the coverage and width of 95% confidence intervals for, and the power of, three within- and between-group contrasts were examined.

Results: In the group with MCAR, negligible bias was observed in all methods, coverage was close to 95%, and little difference was seen in power among methods. In the group with MAR dropout, independent and paired TT and RMA analyses displayed increasing bias and decreasing coverage and power for increasing levels of dropout. The paired TT also produced the widest confidence intervals on average, with the greatest variability. GEE displayed slightly lower coverage and higher power than LME and CP models, but bias and precision were further comparable to LME and CP. LME and CP models produced unbiased results and close to 95% coverage, even in the case of 40% MAR dropout.

Conclusions: As expected, LME and CP models performed best in terms of bias and coverage even in the case of higher levels of MAR data. Paired TT and RMA produce biased results and poor coverage and precision in the presence of MAR data.