Methodological progress note: Pilot randomized controlled trials

Abstract

Definitive randomized controlled trials (RCTs) are the cornerstone of evidence-based medicine but can be complicated, protracted, and expensive. Given the challenges of large-scale trials, pilot trials serve as a crucial initial step, allowing for refinement and validation before embarking on the definitive RCT.¹ They are a crucial element of good study design and, while conducting a pilot RCT does not guarantee success of the definitive RCT, it increases the likelihood of successful trial completion.² More than US$100 billion is invested annually in biomedical research but often this research is conducted wastefully from poor study design and/or study procedures.³ Conducting a well-designed pilot RCT before launching an expensive, time-consuming definitive trial can minimize research waste and improve study conduct.

Small RCTs cannot be branded pilot or feasibility trials to justify a small sample size. Pilot RCTs have a very specific purpose and inform future trial conduct.⁴ Indeed, research models, including the Canadian Critical Care Trials Group programmatic model, the UK Medical Research Council, and the Australian Clinical Trials Alliance, highlight the importance of pilot RCTs as an integral and necessary step in interventional clinical research (Figure 1). Early piloting of research methods and interventions is important in evaluating feasibility and acceptability before the definitive RCT.

The importance of pilot trials has been acknowledged for decades⁵ with trial methods evolving over time. It is within this context that we will discuss pilot RCTs used to inform larger definitive RCTs. We will situate pilot trial methods within a larger research framework and propose important concepts in design and reporting.

The terms “pilot” and “feasibility” trial are used interchangeably by some, but others purport that each type of trial has unique characteristics and therefore define them separately. Whitehead et al.⁴ proposed that pilot trials are a type of feasibility trial with some distinguishing elements: (i) stricter methodology (closely following the definitive study design); (ii) intended to lead to further work; (iii) a smaller version of the larger study; and (iv) focuses on trial processes. This delineation suggests that a pilot RCT is a specific subset of feasibility trial. Henceforth, we adopt the term “pilot.”

Pilot RCTs allow researchers to test and establish feasibility of the study protocol, study processes, data collection, and intervention fidelity and acceptability.^{2, 4, 6} Table 1 details trial elements tested by a pilot RCT.

An important indicator of trial feasibility is the ability to recruit the required numbers of participants, using inclusion/exclusion criteria, from the sample population. Recruitment to RCTs can be challenging and investigators are frequently unable to achieve the predetermined participant numbers.⁶ Investigators' over-estimation of the size of the pool from which to sample ultimately leads to under-recruitment, negates the trial's ability to answer the proposed research question/s, wastes resources and can be unethical, and often results from foregoing a rigorous pilot trial process. Funders also value the importance of testing feasibility before embarking on definitive RCTs. For example, in criteria from the National Health and Medical Research Council of Australia, research projects must be “highly feasible with all required techniques and resources established.”

Traditionally, pilot RCTs have been separate (i.e., external) to the definitive RCT. External pilot RCTs are a stand-alone trial completed before the definitive RCT: data are analyzed, results are generated, and thus inform feasibility of progressing to a definitive trial.⁷ External pilot RCTs are particularly useful if uncertainties exist around intervention fidelity or novel trial aspects (such as new clinical settings).⁸ However, data from external pilot RCTs are not included in the larger trial, published separately and therefore could be seen as “wasted.”⁴ Furthermore, attracting funding for external pilot RCTs may be difficult, particularly when study outcomes are unrelated to patient benefit, but rather focus on feasibility.

Over the last decade, more researchers are using internal or embedded pilot RCTs as an innovative and efficient way to assess protocol and intervention feasibility and acceptability.⁹ Cooper et al. suggest this is useful when overall feasibility has largely been established, and evidence on recruitment, randomization, and attrition rates are required.¹⁰ Internal pilot RCTs are a phase within the definitive RCT and planning occurs alongside the main trial. Clear prespecified progression criteria (see upcoming section) are used to determine progression to a definitive RCT,⁹ with this decision ideally made by an external panel. If appropriate, and no significant changes are required to outcome definitions or interventions, participants' data from the internal pilot can be used in the final analysis of the definitive trial.⁸ Advantages include seamless integration into the definitive trial, ability to make real-time adjustments and efficiency through minimizing duplication and optimizing resources.⁹

Kleidon et al.¹⁴ use these criteria in their external pilot to determine the feasibility of a larger-scale RCT. In this case, all feasibility criteria were met, and the trial progressed to a definitive RCT¹⁸ (Table 2). Feasibility targets will be highly individual from study to study and must consider factors such as funding, number of recruiting sites, and incidence of the condition being studied.

Clear guidance on reporting pilot RCTs is provided in the Consolidated Standards of Reporting Trials (CONSORT) 2010 statement: extension to randomized pilot and feasibility trials¹ which utilizes checklists for study abstract and body and a template flowchart to improve transparency and quality of reporting. The CONSORT statement mentions that, while it does not apply specifically to internal pilot RCTs, it may be applicable or modifiable in part for internal pilot RCTs. It is important to register pilot RCTs with a Clinical Trials Registry so study design is transparent before publication, thus reducing reporting bias.¹ Publishing external pilot RCTs is an important step in dissemination of trial feasibility⁶ and has been aided by the launch of BioMed Central Pilot and Feasibility Studies (https://pilotfeasibilitystudies.biomedcentral.com/) whose core business includes publication of pilot and feasibility studies. For definitive RCTs with an internal pilot, feasibility outcomes from the pilot phase in addition to clinical outcomes (which include data from the internal pilot if no major modifications were made to population, intervention or outcomes) would be reported.

The solid foundation provided by well-designed pilot trials is now broadly recognized. Pilot RCTs afford a preparatory phase for learning before expansion, mitigating the risk of logistical impracticalities leading to failure in subsequent large-scale RCTs. No large publicly funded RCT should be conducted without thorough piloting of interventions, study processes, and study procedures.

Amanda Corley declares her employer, on her behalf, has received unrestricted investigator-initiated research grants, from Cardinal Health, 3M, and Eloquest (unrelated to the current project) and a consultancy payment from Wolters Kluwer for review of clinical practice guidelines. Nicole Marsh declares her employer has received on her behalf speaker fees from 3M and Medline; investigator-initiated research grants from Biolife, 3M, Eloquest, and Cardinal Health; and a consultancy payment from 3M (unrelated to the current project). Samantha Keogh declares her employer, on her behalf, has received monies from BD Medical and ITL Biomedical for Educational consultancies (unrelated to the current project).