Development and validation of a quality appraisal tool for case reports in traditional Chinese medicine using the Delphi method

Abstract

Objective

To develop a quality appraisal tool for case reports in traditional Chinese medicine (TCM) based on their characteristics.

Methods

An extensive literature search was conducted in Chinese Biomedical Literature Database (CBM), China National Knowledge Infrastructure (CNKI), and China Science and Technology Journal Database (CSTJ), focusing on expert consensus statements and checklists for TCM case reports. Relevant items were extracted, and a Delphi method involving 34 experts was used in two rounds to rate each item on a 5-point Likert scale. Items were screened based on measures of central tendency and coordination (including total score, mean score, percentage of items rated as unimportant, and coefficient of variation). The weighted average method was used to determine item weights and construct the appraisal tool. Internal consistency was assessed using Cronbach’s α coefficient. The finalized tool was pilot-tested by two reviewers independently appraising 20 case reports, with an additional four reviewers evaluating 5 of these cases to compare inter-rater consistency.

Results

A total of 9 513 articles were retrieved, and 96 items from 25 articles were extracted. After two rounds of the Delphi method, 27 items across 10 domains were retained. The Cronbach’s α coefficient was 0.72 in the first round (acceptable range), and 0.96 in the second round, indicating strong internal consistency. The tool was piloted by six reviewers, achieving a kappa value of 0.663 and a Kendall’s coefficient of concordance of 0.845, demonstrating high consistency among reviewers.

Conclusion

The developed TCM case report quality appraisal tool, consisting of 27 items in 10 domains, offers a scientific and reliable means of assessing the quality of TCM case reports. The tool showed high consistency and practical utility, and its application is expected to enhance the standardization, scientific rigor, and evidence quality of TCM case reports, facilitating the integration of traditional medical knowledge with modern evidence-based standards.