Factors that can affect the external validity of randomised controlled trials.

Abstract

Randomised controlled trials (RCTs) must be internally valid (i.e., design and conduct must eliminate the possibility of bias), but to be clinically useful, the result must also be relevant to a definable group of patients in a particular clinical setting (i.e., they must be externally valid). Lack of external validity is the most frequent criticism by clinicians of RCTs, systematic reviews, and guidelines, and is one explanation for the widespread underuse in routine practice of many treatments that have been shown to be beneficial in trials and are recommended in guidelines [1]. Yet medical journals, funding agencies, ethics committees, the pharmaceutical industry, and governmental regulators seem to give external validity a low priority. Admittedly, whereas the determinants of internal validity are intuitive and can generally be worked out from first principles, understanding of the determinants of the external validity of an RCT requires clinical rather than statistical expertise, and often depends on a detailed understanding of the particular clinical condition under study and its management in routine clinical practice. However, reliable judgments about the external validity of RCTs are essential if treatments are to be used correctly in as many patients as possible in routine clinical practice. The results of RCTs or systematic reviews will never be relevant to all patients and all settings, but they should be designed and reported in a way that allows clinicians to judge to whom the results can reasonably be applied. Table 1 lists some of the important potential determinants of external validity, each of which is reviewed briefly below. Many of the considerations will only be relevant in certain types of trials, for certain interventions, or in certain clinical settings, but they can each sometimes undermine external validity. Moreover, the list is not exhaustive and requires more detailed annotation and explanation than is possible in this short review. Table 1 Main Issues That Can Affect External Validity and Should Be Addressed in Reports of the Results of Randomised Controlled Trials or Systematic Reviews and Considered by Clinicians Some of the issues that determine external validity are relevant to the distinction between pragmatic trials and explanatory trials [2], but it would be wrong to assume that pragmatic trials necessarily have greater external validity than explanatory trials. For example, broad eligibility criteria, limited collection of baseline data, and inclusion of centres with a range of expertise and differing patient populations have many advantages, but they can also make it very difficult to generalise the overall average effect of treatment to a particular clinical setting.