Meta-analysis of clinical trials in the 2020s and beyond: a paradigm shift needed.

Jonathan J Shuster
{"title":"Meta-analysis of clinical trials in the 2020s and beyond: a paradigm shift needed.","authors":"Jonathan J Shuster","doi":"","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>A peer-reviewed meta-analysis methods article mathematically proved that mainstream random-effects methods, \"weights inversely proportional to the estimated variance,\" are flawed and can lead to faulty public health recommendations. Because the arguments causing this off-label (unproven) use of mainstream practices were subtle, changing these practices will require much clearer explanations that can be grasped by clinical and translational scientists. There are five assumptions underlying the mainstream's derivation of its statistical properties. This paper will demonstrate that if the first is true, it follows that the last two are false. Ratio estimation, borrowed from classical survey sampling, provides a rigorous alternative. Papers reporting results rarely fully disclose these assumptions. This is analogous to watching TV ads with the sound muted. You see high quality of life and do not hear about the complications. This article is a poster child for translational science, as it takes a theoretical discovery from the biostatistical world, translates it into language clinical scientists can understand, and thereby can change their research practice.</p><p><strong>Aim: </strong>This article is aimed at future applications of meta-analysis of complete collections of randomized clinical trials. It leaves it to past authors as to whether to reanalyze their data. No blame for past use is assessed.</p><p><strong>Methods: </strong>By treating the individual completed studies in the meta-analysis as a random sample from a conceptual universe of completed studies, we use ratio estimation to obtain estimates of relative risk (ratio of failure rates treatment: control) and mean differences, projecting our sample value to estimate the universe's value.</p><p><strong>Results: </strong>Two examples demonstrate that the mainstream methods likely adversely impacted major treatment options. A third example shows that the key mainstream presumption of independence between the study weights and study estimates cannot be supported.</p><p><strong>Conclusion: </strong>There is no rationale for ever using the mainstream for meta-analysis of randomized clinical trials.</p><p><strong>Relevance for patients: </strong>Future meta-analysis of clinical trials should never employ mainstream methods. Doing so could lead to potentially harmful public health policy recommendations. Clinical researchers need to play a primary role to assure good research practices in meta-analysis.</p>","PeriodicalId":15482,"journal":{"name":"Journal of Clinical and Translational Research","volume":"9 4","pages":"246-252"},"PeriodicalIF":0.0000,"publicationDate":"2023-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/8c/bf/jclintranslres-2023-9-4-246.PMC10411977.pdf","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Clinical and Translational Research","FirstCategoryId":"1085","ListUrlMain":"","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

Background: A peer-reviewed meta-analysis methods article mathematically proved that mainstream random-effects methods, "weights inversely proportional to the estimated variance," are flawed and can lead to faulty public health recommendations. Because the arguments causing this off-label (unproven) use of mainstream practices were subtle, changing these practices will require much clearer explanations that can be grasped by clinical and translational scientists. There are five assumptions underlying the mainstream's derivation of its statistical properties. This paper will demonstrate that if the first is true, it follows that the last two are false. Ratio estimation, borrowed from classical survey sampling, provides a rigorous alternative. Papers reporting results rarely fully disclose these assumptions. This is analogous to watching TV ads with the sound muted. You see high quality of life and do not hear about the complications. This article is a poster child for translational science, as it takes a theoretical discovery from the biostatistical world, translates it into language clinical scientists can understand, and thereby can change their research practice.

Aim: This article is aimed at future applications of meta-analysis of complete collections of randomized clinical trials. It leaves it to past authors as to whether to reanalyze their data. No blame for past use is assessed.

Methods: By treating the individual completed studies in the meta-analysis as a random sample from a conceptual universe of completed studies, we use ratio estimation to obtain estimates of relative risk (ratio of failure rates treatment: control) and mean differences, projecting our sample value to estimate the universe's value.

Results: Two examples demonstrate that the mainstream methods likely adversely impacted major treatment options. A third example shows that the key mainstream presumption of independence between the study weights and study estimates cannot be supported.

Conclusion: There is no rationale for ever using the mainstream for meta-analysis of randomized clinical trials.

Relevance for patients: Future meta-analysis of clinical trials should never employ mainstream methods. Doing so could lead to potentially harmful public health policy recommendations. Clinical researchers need to play a primary role to assure good research practices in meta-analysis.

21世纪20年代及以后临床试验的荟萃分析:需要范式转变。
背景:一篇同行评议的荟萃分析方法文章用数学方法证明了主流随机效应方法(“权重与估计方差成反比”)是有缺陷的,可能导致错误的公共卫生建议。由于导致这种标签外(未经证实)使用主流实践的争论是微妙的,改变这些实践将需要临床和转化科学家能够掌握的更清晰的解释。主流经济学对其统计特性的推导有五个假设。本文将证明,如果第一个为真,则后两个为假。比率估计,借鉴了经典的调查抽样,提供了一个严格的选择。报告结果的论文很少完全披露这些假设。这类似于观看静音的电视广告。你可以看到高质量的生活,而不会听到并发症。这篇文章是翻译科学的典范,因为它将生物统计学领域的理论发现转化为临床科学家可以理解的语言,从而改变他们的研究实践。目的:本文旨在探讨随机临床试验全集meta分析的未来应用。它把是否重新分析他们的数据留给了过去的作者。过去的使用没有责任被评估。方法:通过将meta分析中的单个已完成研究作为已完成研究概念性宇宙中的随机样本,我们使用比率估计来获得相对风险(治疗失败率与对照组的比率)和平均差异的估计,投影我们的样本值来估计宇宙的值。结果:两个例子表明,主流方法可能对主要治疗方案产生不利影响。第三个例子表明,研究权重和研究估计之间独立的关键主流假设无法得到支持。结论:在随机临床试验中使用主流meta分析是没有理由的。与患者的相关性:临床试验的未来荟萃分析不应采用主流方法。这样做可能导致潜在有害的公共卫生政策建议。临床研究人员需要在meta分析中发挥主要作用,以确保良好的研究实践。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信