Outcomes reporting in clinical trials of Chinese herbal medicine on ulcerative colitis: A systematic review

Ulcerative colitis (UC) is a prevalent type of inflammatory bowel disease (IBD) characterized by inflammation and ulceration in the rectum and colon.¹ The optimal therapeutic effect of current treatment strategies for UC may be unattainable; even surgery may be followed by ongoing morbidity. Pharmacological therapies, mainly including aminosalicylates, steroids, immunosuppressants, etc., are used to control the acute onset of UC, heal the mucosa, and prevent complications.² In clinical practice, some patients, however, may experience a gradual loss of response to the therapy while others may show intolerance to the adverse effects of drugs.³ Consequently, an increasing number of UC patients (21%–60%) prefer to seek additional help from Chinese herbal medicine (CHM). While numerous clinical studies have demonstrated the efficacy of CHM therapies in relieving symptoms, enhancing the therapeutic effects of chemical drugs, and reducing side effects and recurrence rates in UC patients, recommending CHM interventions for UC treatment remains cautious due to significant issues related to the choice and reporting of outcome measures.^4-7 The lack of agreed-upon and standardized evaluation criteria, such as tongue and pulses in Chinese medicine (CM), contributes to considerable variation in outcome measurement and reporting among studies, making comparisons challenging.⁸ Therefore, we aim to summarize existing endpoint definitions and measurement tools, and inspect the efficacy and safety outcomes reported in randomized controlled trials (RCTs) of CHM in adults with UC. Given that some UC patients in China receive integrative Chinese and Western Medicine (ICWM) therapy over CHM or Western Medicine (WM) alone, the development of a core outcome set (COS) for CHM studies of UC is crucial. Such a COS would help reduce outcome heterogeneity, enhance study quality, and contribute to generating robust evidence for innovative UC therapies, ultimately fostering international recognition in the field.

This study included RCTs that investigated CHM, including single herbs, formulas, or both as interventions to treat UC. Accordingly, RCTs published in English or Chinese from January 1, 2011 to December 31, 2022 were limited to adults with UC diagnosis based on clear diagnostic criteria or references, but without limitations in control groups or outcomes. A systematic search was conducted in six databases: All EBM Reviews (Ovid), Allied and Complementary Medicine (Ovid), Embase and Ovid MEDLINE(R) (Ovid), CNKI, VIP, and Wanfang. Detailed inclusion and exclusion criteria and search strategy are presented in Supplementary File 1. Two reviewers were independently involved in reviewing the titles and abstracts, full text of the selected studies based on the criteria. Any disagreements between the reviewers were resolved through discussion or consultation with the third reviewer. A data extraction form was predesigned, including basic information (e.g., publication year, study design, objective, sample size, etc.); details of participants, interventions, and controls; and outcomes information (e.g., numbers, names, category of primary or secondary outcomes, measurement methods and time point, safety outcomes, CM relevant outcomes, and classifications of outcome domain). Data extraction of each record was conducted by two reviewers independently and a double-check procedure was implemented by a third reviewer. Any questions were resolved by the senior reviewer. Specific items with extraction rules are presented in Supplementary Files 2 and 3. Before conducting data analysis, all outcomes were standardized by combining similar expressions or grouping them into categories while maintaining their original meanings. For example, terms like “The Inflammatory Bowel Disease Questionnaire” and “IBDQ” were consolidated under “Quality of life (IBDQ),” and indicators such as “Improvement of diarrhoea/abdominal pain/mucus in the stool/bloody purulent stool” and “single symptom score” were grouped as “Improvement of clinical symptoms.” Subsequently, to ensure transparency, we documented both the original and standardized names of the extracted outcomes in Supplementary Files 4 and 5. Descriptive statistical analyses were conducted, with discrete data reported as frequencies or percentages and continuous data presented as mean and range. Data extraction and management were performed using Microsoft Office Excel 2016, while data analysis was carried out using SPSS 25.0.

As a result, 1247 RCTs were included for analysis, comprising 0.5% (6 articles) in English and 99.5% (1241 articles) in Chinese (Supplementary File 6). The characteristics of the included studies are provided in Supplementary File 7. A total of 3908 outcomes were originally extracted from 1247 included articles. After normalization and elimination of repeatability, 125 normalized outcomes were identified, of which the average number of outcomes in each trial was nearly 3 (n = 3.1; range from 1 to 11). Only 14 trials (1.1%) distinguished primary or secondary outcomes in the reporting. Referring to the core indicator set of IBD and UC, we categorized outcomes into ten domains, including clinical composite outcomes, endoscopy-related outcomes, histologic outcomes, biomarker outcomes, patient-reported outcomes, CM symptoms/patterns outcomes, time-related outcomes, relapse-related outcomes, safety outcomes, and others.^9-11 The notable aspect of this outcome domain is the specific categorization of CM-related indicators into a distinct domain for RCTs with CHM interventions for UC. We listed a detailed analysis of these outcomes in Table 1 and reported their measurement time points in Supplementary File 8. A brief summary is provided as follows

The three most frequently observed outcome domains were clinical composite outcomes 48.2% (1885/3908), safety outcomes 13.0% (509/3908), and biochemical-related outcomes 11.3% (440/3908). Regarding the duration of treatment time points, it was presented as 6 (4, 8) weeks, 6 (4, 8) weeks, and 8 (4, 8) weeks for the above three outcomes, respectively. Firstly, 40.6% of clinical composite outcomes with detailed measurement methods were reported, and the “Rate of Clinical Efficacy” was the most commonly used. This category of outcomes encompassed a total of 16 specific names or measurement methods. However, it should be noted that 59.4% of RCTs that adopted clinical composite outcomes did not report the detailed measurement methods; namely, they only provided a composite outcome name without specifying the assessed factors. Secondly, safety outcomes included a total of 8 specific names or measurement methods, and all of them had a frequency equal to or greater than 2. Compared to the 6 weeks (4 to 8 weeks) at the end of treatment, 29 safety outcomes were assessed over a treatment period of 4 weeks (0 to 8 weeks), while 144 were included in follow-up assessments spanning 16 weeks (12 to 26 weeks). Thirdly, a total of 43 specific names or measurement methods of biochemical-related outcomes were identified, with only 24 items having a frequency equal to or greater than 2. Serum inflammatory markers (e.g., CRP, IL-2, IL-7, IL-17, TNF-α, etc.) were the most commonly used indicators.

Besides, a total of 288 outcomes were categorized into “CM symptoms/patterns outcomes.” The most used indicators include “Total CM Symptoms/Pattern Scores” and “Single CM Symptom Scores,” along with 81 outcomes were assessed over a treatment period of 4 weeks (3 to 5 weeks) and 72 outcomes were included in follow-up assessments spanning 12 weeks (8 to 13 weeks). Moreover, other outcome domains comprised “relapse-related outcomes” (238/3908, 6.1%), “endoscopic-related outcomes” (210/3908, 5.4%), “patient-reported outcomes” (144/3908, 3.7%), “histologic-related outcomes” (94/3908, 2.4%), “time-related outcomes” (80/3908, 2.0%), and “Others” category (20/3908, 0.5%). It is noted that no relapse-related outcomes were assessed during the treatment, compared to the 6-week (4 to 8 weeks) evaluation time point at the end of treatment; 171 outcomes were included in follow-up assessments spanning 26 weeks (19 to 52 weeks). Outcomes included in the 1247 RCTs that cannot be classified in the above domains were grouped into the “Others” category, accounting for only 0.5% of the 3908 outcomes. These outcomes included Hospital Fees (3/20, 15%), Hemorheology (3/20, 15%), and Physician's Global Assessment (3/20, 15%), etc.

During the implementation of this study, our objective was to review the application of outcomes in CHM interventional RCTs for UC over the last 10 years. Given that a COS for IBD was newly published in 2023, we set the cutoff for this review at the year 2022. Future plans include conducting a comparative analysis to assess whether any improvements in the design of the outcome application will be identified after the issuance of this COS standard. In conclusion, several critical deficiencies were identified in this review. Firstly, various heterogeneities exist in the usage of outcomes, including non-standard names, non-consensus measurement methods, and references, as well as the absence of reporting on these items. Secondly, most trials did not clearly identify the primary outcome(s), leading to confusion in the application of outcome(s) design, selection, and assessment. Thirdly, CM-related outcome(s) were infrequently used in CHM studies, which limited the exploration of CM-relevant outcomes for the outcome library/pool for the development of COS. The accurate identification of CM patterns is crucial for ensuring clinical efficacy, as misdiagnoses could lead to ineffective interventions and potential adverse effects.^{12, 13} Therefore, further research, such as surveys, semistructured interviews, and expert consultations, is needed to evaluate the feasibility and appropriateness of establishing COS for CHM studies on UC. Another effective pathway may be to develop consensus-based recommendations on the design and assessment of different types of CM-related outcomes, as a supplement reference on available COS for IBD for ICWM studies.

This work is supported by Chinese Medicine Development Fund, Hong Kong, China (23B2/027A_R1); Center for Evidence Based Traditional Chinese Medicine, CCEBTM (2020YJSZX-5); National Natural Science Foundation of China (No. 81704198); and Donation funding of Vincent V.C. Woo Chinese Medicine Clinical Research Institute. The funders had no role in the design of study, in the collection, analysis, and interpretation of data, nor the writing of the manuscript.