Methodological Challenges in Pilot Trials of Herbal Medicine: Barriers to Evidence-Based Practice.

Abstract

Objective: The growing popularity of herbal medicine (HM) underscores the need for high-quality clinical trials to support its evidence-based integration. Pilot trials are essential for addressing methodological challenges in this field. This study evaluates the design quality, feasibility, and reporting of HM pilot trials, with a focus on their capacity to inform future full-scale studies.

Study design and setting: A comprehensive collection of herbal medicine HM pilot trials was conducted using PubMe, Web of Science and Embase, based on predefined inclusion criteria. Data were extracted on trial characteristics, reporting quality, and progression to full-scale studies. To gather additional information on follow-up studies, authors of selected trials were contacted directly by email. Adherence to CONSORT guidelines for pilot trials was evaluated, and Poisson regression was applied to identify factors influencing reporting completeness.

Results: A total of 123 HM pilot trials were reviewed, predominantly from Asia (78.1%). Trials most commonly addressed respiratory (14.6%), nervous (14.6%), and reproductive systems (13.0%). Key gaps in reporting included feasibility assessments (13.1%), sample size rationale (47.2%), and randomization methods (35.8%). Herbal medicine-specific details, including ingredient processing, quality control, and safety assessments, were inconsistently reported. Among the trials, 4 (3.3%) progressed to full-scale studies. Factors such as trial registration (IRR = 1.20, 95% CI: 1.11-1.30) and protocol publication (IRR = 1.16, 95% CI: 1.08-1.24) were positively associated with reporting completeness. Moreover, an analysis of the origin of herbal medicines revealed that modern HM trials were 4.7 times more likely to progress to full-scale studies compared to traditional HM trials (OR = 4.70, 95% CI: 0.37-252.91), although the result did not reach statistical significance (p = 0.300).

Conclusion: Herbal medicine pilot trials, as they stand, are not yet equipped to reliably guide full-scale studies. Core issues in methodological rigor, particularly in feasibility assessment, sample size justification, and randomization processes, limit their effectiveness and integration into evidence-based practice. A dedicated checklist that merges pilot study standards with the unique needs of HM trials is essential.