Paired or Pooled Analyses in Continuing Medical Education, Which One is Better?

Abstract

In data analyses, pairing participant responses is often thought to yield the purest results. However, ensuring all participants answer all questions can be challenging. Concerns exist that pooling all responses together may diminish the robustness of a statistical analysis, but the practical insights may still exist. Data from a live, in-person, continuing education series for health professionals was analysed. For each topic, identical questions were asked prior to the educational content (pre), immediately following the content (post), and on a rolling 4 to 6 week follow-up survey (follow-up). For each educational topic, responses were matched by participant for a paired analysis and aggregated for a pooled analysis. A paired analysis was done for matched responses on pre vs post and pre vs follow-up questions. A pooled analysis was done for the aggregate responses on pre vs post and pre vs follow-up questions. Responses from 55 questions were included in the analysis. In both the paired and pooled pre vs post analyses, all questions yielded a statistically significant improvement in correct responses. In the paired pre vs follow-up analysis, 59% (n = 33) of questions demonstrated a statistically significant improvement in correct responses, compared to 62% (n = 35) in the pooled pre vs follow-up analysis. Paired and pooled data yielded similar results at the immediate post-content and follow-up time periods.