Reasons for caution when evaluating health care interventions using non-randomised study designs.

Abstract

The key difference between randomised (RCTs) and non-randomised studies (NRS) is their susceptibility to selection bias. Unlike RCTs, groups in non-randomised cohort studies are unlikely to be balanced because of the reasons leading patients to receive one or another treatment, giving rise to "confounding by clinical indication" (CCI). Researchers can try to minimise the susceptibility of NRS to selection bias both at the design stage, e. g. by matching participants on key prognostic factors, and during data analysis, e. g. by regression modeling. Nevertheless, because of i) logistical difficulties in matching, ii) imperfect knowledge about the relationships between prognostic factors and between prognostic factors and outcome, and iii) because of measurement limitations, it is inevitable that estimates of effect size derived from NRS will be confounded to some extent. Researchers and users of evidence alike need to be aware of the consequences of residual confounding. CCI need not necessarily lead to systematic bias in favour of one treatment but, if CCI acts in an unpredictable way, it will still give rise to additional, non-statistical "uncertainty bias" around the estimate of effect size.