Daniel Tobias Michaeli, Thomas Michaeli, Sebastian Albers, Julia Caroline Michaeli
{"title":"Clinical trial design and treatment effects: a meta-analysis of randomised controlled and single-arm trials supporting 437 FDA approvals of cancer drugs and indications.","authors":"Daniel Tobias Michaeli, Thomas Michaeli, Sebastian Albers, Julia Caroline Michaeli","doi":"10.1136/bmjebm-2023-112544","DOIUrl":"10.1136/bmjebm-2023-112544","url":null,"abstract":"<p><strong>Objectives: </strong>This study aims to analyse the association between clinical trial design and treatment effects for cancer drugs with US Food and Drug Administration (FDA) approval.</p><p><strong>Design: </strong>Cross-sectional study and meta-analysis.</p><p><strong>Setting: </strong>Data from Drugs@FDA, FDA labels, ClincialTrials.gov and the Global Burden of Disease study.</p><p><strong>Participants: </strong>Pivotal trials for 170 drugs with FDA approval across 437 cancer indications between 2000 and 2022.</p><p><strong>Main outcome measures: </strong>Treatment effects were measured in HRs for overall survival (OS) and progression-free survival (PFS), and in relative risk for tumour response. Random-effects meta-analyses and meta-regressions explored the association between treatment effect estimates and clinical trial design for randomised controlled trials (RCTs) and single-arm trials.</p><p><strong>Results: </strong>Across RCTs, greater effect estimates were observed in smaller trials for OS (ß=0.06, p<0.001), PFS (ß=0.15, p<0.001) and tumour response (ß=-3.61, p<0.001). Effect estimates were larger in shorter trials for OS (ß=0.08, p<0.001) and PFS (ß=0.09, p=0.002). OS (ß=0.04, p=0.006), PFS (ß=0.10, p<0.001) and tumour response (ß=-2.91, p=0.004) outcomes were greater in trials with fewer centres. HRs for PFS (0.54 vs 0.62, p=0.011) were lower in trials testing the new drug to an inactive (placebo/no treatment) rather than an active comparator. The analysed efficacy population (intention-to-treat, per-protocol, or as-treated) was not consistently associated with treatment effects. Results were consistent for single-arm trials and in multivariable analyses.</p><p><strong>Conclusions: </strong>Pivotal trial design is significantly associated with measured treatment effects. Particularly small, short, single-centre trials testing a new drug compared with an inactive rather than an active comparator could overstate treatment outcomes. Future studies should verify results in unsuccessful trials, adjust for further confounders and examine other therapeutic areas. The FDA, manufacturers and trialists must strive to conduct robust clinical trials with a low risk of bias.</p>","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":null,"pages":null},"PeriodicalIF":9.0,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140955606","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Vibration of effects resulting from treatment selection in mixed-treatment comparisons: a multiverse analysis on network meta-analyses of antidepressants in major depressive disorder.","authors":"Constant Vinatier, Clement Palpacuer, Alexandre Scanff, Florian Naudet","doi":"10.1136/bmjebm-2024-112848","DOIUrl":"10.1136/bmjebm-2024-112848","url":null,"abstract":"<p><strong>Objective: </strong>It is frequent to find overlapping network meta-analyses (NMAs) on the same topic with differences in terms of both treatments included and effect estimates. We aimed to evaluate the impact on effect estimates of selecting different treatment combinations (ie, network geometries) for inclusion in NMAs.</p><p><strong>Design: </strong>Multiverse analysis, covering all possible NMAs on different combinations of treatments.</p><p><strong>Setting: </strong>Data from a previously published NMA exploring the comparative effectiveness of 22 treatments (21 antidepressants and a placebo) for the treatment of acute major depressive disorder.</p><p><strong>Participants: </strong>Cipriani <i>et al</i> explored a dataset of 116 477 patients included in 522 randomised controlled trials.</p><p><strong>Main outcome measures: </strong>For each possible treatment selection, we performed an NMA to estimate comparative effectiveness on treatment response and treatment discontinuation for the treatments included (231 between-treatment comparisons). The distribution of effect estimates of between-treatment comparisons across NMAs was computed, and the direction, magnitude and statistical significance of the 1st and 99th percentiles were compared.</p><p><strong>Results: </strong>4 116 254 different NMAs concerned treatment response. Among possible network geometries, 172/231 (74%) pairwise comparisons exhibited opposite effects between the 1st and 99th percentiles, 57/231 (25%) comparisons exhibited statistically significant results in opposite directions, 118 of 231 (51%) comparisons derived results that were both significant and non-significant at 5% risk and 56/231 (24%) treatment pairs obtained consistent results with only significant differences (or only non-significant differences) at 5% risk. Comparisons based on indirect evidence only were associated with greater variability in effect estimates. Comparisons with small absolute values observed in the complete NMA more frequently obtained statistically significant results in opposite directions. Similar results were observed for treatment discontinuation.</p><p><strong>Conclusion: </strong>In this multiverse analysis, we observed that the selection of treatments to be included in an NMA could have considerable consequences on treatment effect estimations.</p><p><strong>Trial registration: </strong>https://osf.io/mb5dy.</p>","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":null,"pages":null},"PeriodicalIF":9.0,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11503165/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141070460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mohammad Hassan Murad, Zhen Wang, Yngve Falck-Ytter
{"title":"Facilitating GRADE judgements about the inconsistency of effects using a novel visualisation approach.","authors":"Mohammad Hassan Murad, Zhen Wang, Yngve Falck-Ytter","doi":"10.1136/bmjebm-2024-113038","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-113038","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":null,"pages":null},"PeriodicalIF":9.0,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142280158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mariska Tuut, Jochen Cals, Jesse Jansen, Jako S Burgers
{"title":"Developing guideline recommendations about tests: educational examples of test-management pathways","authors":"Mariska Tuut, Jochen Cals, Jesse Jansen, Jako S Burgers","doi":"10.1136/bmjebm-2024-112984","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-112984","url":null,"abstract":"Recommendations about healthcare related testing in guidelines are common. Tests can be used for several purposes: screening, surveillance, risk classification, diagnosis, staging, treatment triage, determination of prognosis and monitoring/follow-up.1 The development of testing recommendations in guidelines is challenging, especially because the benefit of a test not only depends on test characteristics, such as sensitivity and specificity, but also on population characteristics and test consequences, such as management.2–4 Furthermore, the role of a new test in comparison to the existing testing scenario should be defined, since this influences the interpretation of the new test’s value. The following roles of new tests have been identified in the literature: triage, replacement, add-on, and parallel/combined.5 As with treatment, testing can have negative consequences, including physical impairment, psychological distress, disease labelling, and costs.6 There is limited evidence on harms of testing, and healthcare professionals often overestimate its benefits while underestimating its harms.7 This is also true for patients' expectations of testing.8 Additionally, testing occasionally yields unexpected and coincidental findings, which may result in additional testing and treatment. There is a lack of transparency in processing the evidence and considerations that support testing recommendations in guidelines.9 To facilitate the development of test recommendations, we determined the minimum required knowledge for guideline panel members involved, supplementing the competency-based framework available for guideline development.10 11 The concept of the test-management pathway (figure 1) appeared key to understand. Figure 1 Test-management pathway concept. During our developmental study, the need for practical examples of test-management pathways became apparent.10 In our subsequent teach-the-teacher workshop at the 2023 Guideline International Network conference,12 participants requested additional elaboration of pathways for different test outcomes (such as false positives and false negatives) being helpful for explaining the test-management pathway concept to guideline …","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":null,"pages":null},"PeriodicalIF":5.8,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142254700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ryan Muller, Giovanni Ferreira, Geronimo Bejarano, Andrew R Gamble, James Kirk, James Sindone, Joshua R Zadro
{"title":"Do infographics ‘spin’ the findings of health and medical research?","authors":"Ryan Muller, Giovanni Ferreira, Geronimo Bejarano, Andrew R Gamble, James Kirk, James Sindone, Joshua R Zadro","doi":"10.1136/bmjebm-2024-113033","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-113033","url":null,"abstract":"Objective To compare the prevalence of ‘spin’, and specific reporting strategies for spin, between infographics, abstracts and full texts of randomised controlled trials (RCTs) reporting non-significant findings in the field of health and medicine and to assess factors associated with the presence of spin. Design Cross-sectional observational study. Data source Publications in top quintile health and medical journals from August 2018 to October 2020 (Journal Citation Reports database). Eligibility criteria Infographics, abstracts and full texts of RCTs with non-significant results for a primary outcome. Main outcome(s) and measure(s) Presence of spin (any spin and spin in the results and conclusions of infographics, abstracts and full texts). Exposure(s) Conflicts of interest, industry sponsorship, trial registration, journal impact factor, spin in the abstract, spin in the full text. Results 119 studies from 40 journals were included. One-third (33%) of infographics contained spin. Infographics were not more likely to contain any spin than abstracts (33% vs 26%, OR 1.4; 95% CI 0.8 to 2.4) or full texts (33% vs 26%, OR 1.4; 95% CI 0.8 to 2.4). Higher journal impact factor was associated with slightly lower odds of spin in infographics and full texts, but not abstracts. Infographics, but not abstracts or full texts, were less likely to contain spin if the trial was prospectively registered. No other significant associations were found. Conclusions Nearly one-third of infographics spin the findings of RCTs with non-significant results for a primary outcome, but the prevalence of spin is not higher than in abstracts and full texts. Given the increasing popularity of infographics to disseminate research findings, there is an urgent need to improve the reporting of research in infographics. Data are available on reasonable request. Original data may be made available by the authors on reasonable request.","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":null,"pages":null},"PeriodicalIF":5.8,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Glyn Elwyn, Marie Anne Durand, Thomas Agoritsas, Martin Mayer
{"title":"How to best convey continuous outcomes in patient decision aids","authors":"Glyn Elwyn, Marie Anne Durand, Thomas Agoritsas, Martin Mayer","doi":"10.1136/bmjebm-2024-112871","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-112871","url":null,"abstract":"The work of transforming scientific publications into tools that can support people in comparing treatments, tests and other interventions has been driven by efforts to support shared decision-making. The International Patient Decision Aids Standards (IPDAS) Collaboration has published many articles that guide this process. IPDAS, and others in this field, have considered the challenges of representing complex concepts such as effect sizes, ORs and relative risk rates in terms and formats that are easier to understand by people with varying levels of health literacy and numeracy. The underlying task is how to simplify research results without misleading people, which is essential when communicating healthcare information. Selecting outcome probabilities and comparing data collected from different populations with various study designs would be misleading. Further, only providing relative instead of absolute risk estimates would lead to misinterpretation in most situations, framing that affects both professionals and patients.1 For example, imagine a person with a 4% 10-year risk of breast cancer recurrence: telling that person their risk of breast cancer coming back could be cut in half, or reduced by 50%, using treatment Y (relative risk) is misleading. The more informative approach would be telling them their risk could be reduced from 4% to 2% (absolute risk), a risk that is perhaps even clearer if portrayed in natural frequency (n in 100). This article tackles the challenge of untangling complicated, heterogeneous evidence to deliver comparative information to non-scientists while at the same time avoiding misrepresenting the underlying data. Our focus here is to describe some of the editorial challenges of presenting complex scientific information, particularly when published as continuous outcomes in ways that are accurate yet accessible to a broad public, irrespective of their health literacy and numeracy levels. Those editorial challenges have materialised when developing tools to promote shared decision-making. The …","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":null,"pages":null},"PeriodicalIF":5.8,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Simon Dixon, Kirti Tyagi, Malkeet Singh, Sitanshu Sekhar Kar, Bhavani Shankara Bagepally, Shankar Prinja, Andrew Booth, Chris Carroll, Aamir Sohail, Abha Mehndiratta
{"title":"Development of a competency framework for health technology assessment in India.","authors":"Simon Dixon, Kirti Tyagi, Malkeet Singh, Sitanshu Sekhar Kar, Bhavani Shankara Bagepally, Shankar Prinja, Andrew Booth, Chris Carroll, Aamir Sohail, Abha Mehndiratta","doi":"10.1136/bmjebm-2023-112488","DOIUrl":"https://doi.org/10.1136/bmjebm-2023-112488","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":null,"pages":null},"PeriodicalIF":9.0,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142139263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alison Huffstetler, Kenneth W Lin, Russell P Harris
{"title":"Assessing proposals to update established screening strategies.","authors":"Alison Huffstetler, Kenneth W Lin, Russell P Harris","doi":"10.1136/bmjebm-2024-113025","DOIUrl":"https://doi.org/10.1136/bmjebm-2024-113025","url":null,"abstract":"","PeriodicalId":9059,"journal":{"name":"BMJ Evidence-Based Medicine","volume":null,"pages":null},"PeriodicalIF":9.0,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142124772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}