Do Meta-Analyses of Total Hip Arthroplasty Produce Reliable Results? A Systematic Review and Meta-Epidemiological Study of Statistical Methods.

Abstract

Background: Total hip arthroplasty (THA) is a highly successful orthopedic procedure, with numerous meta-analyses published to optimize its outcomes. However, the reliability of their results and conclusions depends heavily on the use of appropriate statistical methods. Therefore, the aim was to test the reliability of statistical methods in meta-analyses of THA by examining the degree of heterogeneity, the effect of different between-study variance estimators, and the equality of sample size of pooled primary studies.

Methods: The literature was systematically searched in PubMed from January 1, 2022, to December 31, 2023, for meta-analyses on THA. The quality of the meta-analyses was assessed using the revised Measurement Tool to Assess Systematic Reviews (AMSTAR 2). All meta-analyses were recalculated using eight different heterogeneity estimators. The following indicators were considered: inequality of patient numbers, proportion of random-effects and fixed-effects models, heterogeneity with I² value, ratio of effect sizes (RES), ratio of confidence interval width (RCIW), and the number of significant results. Mixed linear regression was then used to analyze whether the effect sizes and CIW were significantly different using different heterogeneity estimators. Finally, all examined meta-analyses were recalculated using the eight heterogeneity estimators and the Hartung-Knapp (HK) adjustment.

Results: Of the 24 meta-analyses examined, 15 reported an outcome using a mean difference and 20 reported an outcome using an odds ratio. The quality assessment identified 10 meta-analyses of high quality, 7 of moderate quality, 4 of low quality, and 3 of critically low quality. The significance of the examined meta-analyses varied considerably depending on the heterogeneity estimators used. In particular, the DerSimonian and Laird and Hunter-Schmidt heterogeneity estimators tended to produce false-positive results. The meta-analyses examined generally did not use HK adjustment. This effect is amplified when combined with the weak DerSimonian and Laird heterogeneity estimator, which were used in almost all examined meta-analyses.

Conclusion: Without HK adjustment, the results depend strongly on the heterogeneity estimator chosen and there is a risk of false positives, especially for the widely used DerSimonian and Laird heterogeneity estimator. For HK adjustment, the choice of heterogeneity estimator seems to play a less important role. We recommend the use of more reliable heterogeneity estimators as well as the HK adjustment as a measure to improve the statistical methodology of meta-analyses. This study highlights the critical need for improved statistical rigor in meta-analyses of THA, ensuring more reliable evidence for clinical decision-making and guideline development.