Research Synthesis Methods最新文献

{"title":"Assessing risk of bias of cohort studies with large language models.","authors":"Danni Xia, Honghao Lai, Weilong Zhao, Jiajie Huang, Jiayi Liu, Ziying Ye, Jianing Liu, Mingyao Sun, Liangying Hou, Bei Pan, Long Ge","doi":"10.1017/rsm.2025.10028","DOIUrl":"10.1017/rsm.2025.10028","url":null,"abstract":"<p><p>This study aims to explore the feasibility and accuracy of utilizing large language models (LLMs) to assess the risk of bias (ROB) in cohort studies. We conducted a pilot and feasibility study in 30 cohort studies randomly selected from reference lists of published Cochrane reviews. We developed a structured prompt to guide the ChatGPT-4o, Moonshot-v1-128k, and DeepSeek-V3 to assess the ROB of each cohort twice. We used the ROB results assessed by three evidence-based medicine experts as the gold standard, and then we evaluated the accuracy of LLMs by calculating the correct assessment rate, sensitivity, specificity, and <i>F</i>1 scores for overall and item-specific levels. The consistency of the overall and item-specific assessment results was evaluated using Cohen's kappa (κ) and prevalence-adjusted bias-adjusted kappa. Efficiency was estimated by the mean assessment time required. This study assessed three LLMs (ChatGPT-4o, Moonshot-v1-128k, and DeepSeek-V3) and revealed distinct performance across eight assessment items. Overall accuracy was comparable (80.8%-83.3%). Moonshot-v1-128k showed superior sensitivity in population selection (0.92 versus ChatGPT-4o's 0.55, <i>P</i> < 0.001). In terms of <i>F</i>1 scores, Moonshot-v1-128k led in population selection (<i>F</i> = 0.80 versus ChatGPT-4o's 0.67, <i>P</i> = 0.004). ChatGPT-4o demonstrated the highest consistency (mean κ = 96.5%), with perfect agreement (100%) in outcome confidence. ChatGPT-4o was 97.3% faster per article (32.8 seconds versus 20 minutes manually) and outperformed Moonshot-v1-128k and DeepSeek-V3 by 47-50% in processing speed. The efficient and accurate assessment of ROB in cohort studies by ChatGPT-4o, Moonshot-v1-128k, and DeepSeek-V3 highlights the potential of LLMs to enhance the systematic review process.</p>","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 6","pages":"990-1004"},"PeriodicalIF":8.0,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12657654/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Novel approaches for random-effects meta-analysis of a small number of studies under normality.","authors":"Yajie Duan, Thomas Mathew, Demissie Alemayehu, Ge Cheng","doi":"10.1017/rsm.2025.10022","DOIUrl":"10.1017/rsm.2025.10022","url":null,"abstract":"<p><p>Random-effects meta-analyses with only a few studies often face challenges in accurately estimating between-study heterogeneity, leading to biased effect estimates and confidence intervals with poor coverage. This issue is especially the case when dealing with rare diseases. To address this problem for normally distributed outcomes, two new approaches have been proposed to provide confidence limits of the global mean: one based on fiducial inference, and the other involving two modifications of the signed log-likelihood ratio test statistic in order to have improved performance with small numbers of studies. The performance of the proposed methods was evaluated numerically and compared with the Hartung-Knapp-Sidik-Jonkman approach and its modification to handle small numbers of studies. The simulation results indicated that the proposed methods achieved coverage probabilities closer to the nominal level and produced shorter confidence intervals compared to those based on existing methods. Two real examples are used to illustrate the proposed methods.</p>","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 6","pages":"922-936"},"PeriodicalIF":8.0,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12657671/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103216","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Authors' reply: Continuity corrections with Mantel-Haenszel estimators in Cochrane reviews.","authors":"Yasushi Tsujimoto, Yusuke Tsutsumi, Yuki Kataoka, Akihiro Shiroshita, Orestis Efthimiou, Toshi A Furukawa","doi":"10.1017/rsm.2025.10013","DOIUrl":"10.1017/rsm.2025.10013","url":null,"abstract":"","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 5","pages":"826-827"},"PeriodicalIF":6.1,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12527533/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Ten practices for successful study coding in research syntheses: Developing coding manuals and coding forms.","authors":"Gena Nelson, Sarah Quinn, Sean Grant, Shaina D Trevino, Elizabeth Day, Maria Schweer-Collins, Hannah Carter, Peter Boedeker, Emily Tanner-Smith","doi":"10.1017/rsm.2025.10019","DOIUrl":"10.1017/rsm.2025.10019","url":null,"abstract":"<p><p>Study coding is an essential component of the research synthesis process. Data extracted during study coding serve as a direct link between the included studies and the synthesis results, allowing reviewers to justify claims about the findings from a set of related studies. The purpose of this tutorial is to provide authors, particularly those new to research synthesis, with recommendations to develop study coding manuals and forms that result in efficient, high-quality data extraction. Each of the 10 easy-to-follow practices is supported with additional resources, examples, or non-examples to help authors develop high-quality study coding materials. With the increase in publication of meta-analyses in recent years across many disciplines, a primary goal of this article is to enhance the quality of study coding materials that authors develop.</p>","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 5","pages":"709-728"},"PeriodicalIF":8.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12527492/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103076","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Subgroup identification using individual participant data from multiple trials: An application in low back pain.","authors":"Cynthia Huber, Tim Friede","doi":"10.1017/rsm.2025.10010","DOIUrl":"10.1017/rsm.2025.10010","url":null,"abstract":"<p><p>Model-based recursive partitioning (MOB) and its extension, metaMOB, are tools for identifying subgroups with differential treatment effects. When pooling data from various trials the metaMOB approach uses random effects to model the heterogeneity of treatment effects. In situations where interventions offer only small overall benefits and require extensive, costly trials with a large participant enrollment, leveraging individual-participant data (IPD) from multiple trials can help identify individuals who are most likely to benefit from the intervention. We explore the application of MOB and metaMOB in the context of non-specific low back pain treatment, using synthetic data based on a subset of the individual participant data meta-analysis by Patel et al. ¹ Our study underscores the need to explore heterogeneity in intercepts and treatment effects to identify subgroups with differential treatment effects in IPD meta-analyses.</p>","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 5","pages":"813-822"},"PeriodicalIF":8.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12527538/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103130","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Hierarchical imputation of categorical variables in the presence of systematically and sporadically missing data.","authors":"Shahab Jolani","doi":"10.1017/rsm.2025.10017","DOIUrl":"10.1017/rsm.2025.10017","url":null,"abstract":"<p><p>Modern quantitative evidence synthesis methods often combine patient-level data from different sources, known as individual participants data (IPD) sets. A specific challenge in meta-analysis of IPD sets is the presence of systematically missing data, when certain variables are not measured in some studies, and sporadically missing data, when measurements of certain variables are incomplete across different studies. Multiple imputation (MI) is among the better approaches to deal with missing data. However, MI of hierarchical data, such as IPD meta-analysis, requires advanced imputation routines that preserve the hierarchical data structure and accommodate the presence of both systematically and sporadically missing data. We have recently developed a new class of hierarchical imputation methods within the MICE framework tailored for continuous variables. This article discusses the extensions of this methodology to categorical variables, accommodating the simultaneous presence of systematically and sporadically missing data in nested designs with arbitrary missing data patterns. To address the challenge of the categorical nature of the data, we propose an accept-reject algorithm during the imputation process. Following theoretical discussions, we evaluate the performance of the new methodology through simulation studies and demonstrate its application using an IPD set from patients with kidney disease.</p>","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 5","pages":"729-757"},"PeriodicalIF":8.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12527547/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Continuity corrections with Mantel-Haenszel estimators in Cochrane reviews.","authors":"A E Ades, Deborah M Caldwell, Sumayya Anwer, Sofia Dias","doi":"10.1017/rsm.2025.10012","DOIUrl":"10.1017/rsm.2025.10012","url":null,"abstract":"","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 5","pages":"823-825"},"PeriodicalIF":6.1,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12527532/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Methodology for mapping reviews, evidence maps, and gap maps.","authors":"Hanan Khalil, Vivian Welch, Matthew Grainger, Fiona Campbell","doi":"10.1017/rsm.2025.25","DOIUrl":"10.1017/rsm.2025.25","url":null,"abstract":"<p><p>Mapping reviews are valuable tools for synthesizing and visualizing research evidence, providing a comprehensive overview of studies within a specific field. Their visual approach enhances accessibility, enabling researchers, policymakers, and practitioners to efficiently identify key findings, trends, and knowledge gaps. These reviews are particularly significant in guiding future research, informing funding decisions, and shaping evidence-based policymaking. In environmental science-similar to health and social sciences-mapping reviews play a crucial role in identifying effective conservation strategies, tracking interventions, and supporting targeted programs.Unlike systematic reviews, which assess intervention effectiveness, mapping reviews focus on broad research questions, aiming to chart the existing evidence on a given topic. They use structured methodologies to identify patterns, gaps, and trends, often employing visual tools to enhance data accessibility. A well-defined scope, guided by inclusion and exclusion criteria, ensures a transparent study selection process. Comprehensive search strategies, often spanning multiple databases, maximize evidence capture. Effective screening, combining automated and manual processes, ensures relevance, while data extraction emphasizes high-level categories such as study design and population demographics. Advanced software tools, including EPPI-Reviewer and MindMeister, support data extraction and visualization, with evidence gap maps highlighting robust areas and research voids.Despite their advantages, mapping reviews present challenges. The categorization and coding of studies can introduce subjective biases, and the process demands substantial resources. Automation and artificial intelligence offer promising solutions, improving efficiency while addressing integration and multilingual limitations. As methodological advancements continue, interdisciplinary collaboration will be essential to fully realize the potential of mapping reviews across scientific disciplines.</p>","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 5","pages":"786-796"},"PeriodicalIF":8.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12527509/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103041","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Tipping point analysis in network meta-analysis.","authors":"Zheng Wang, Thomas A Murray, Wenshan Han, Lifeng Lin, Lianne K Siegel, Haitao Chu","doi":"10.1017/rsm.2025.24","DOIUrl":"10.1017/rsm.2025.24","url":null,"abstract":"<p><p>Network meta-analysis (NMA) enables simultaneous assessment of multiple treatments by combining both direct and indirect evidence. While NMAs are increasingly important in healthcare decision-making, challenges remain due to limited direct comparisons between treatments. This data sparsity complicates the accurate estimation of correlations among treatments in arm-based NMA (AB-NMA). To address these challenges, we introduce a novel sensitivity analysis tool tailored for AB-NMA. This study pioneers a tipping point analysis within a Bayesian framework, specifically targeting correlation parameters to assess their influence on the robustness of conclusions about relative treatment effects. The analysis explores changes in the conclusion based on whether the 95% credible interval includes the null value (referred to as the <i>interval conclusion</i>) and the magnitude of point estimates. Applying this approach to multiple NMA datasets, including 112 treatment pairs, we identified tipping points in 13 pairs (11.6%) for <i>interval conclusion change</i> and in 29 pairs (25.9%) for <i>magnitude change</i> with a threshold at 15%. These findings underscore potential commonality in tipping points and emphasize the importance of our proposed analysis, especially in networks with sparse direct comparisons or wide credible intervals for correlation estimates. A case study provides a visual illustration and interpretation of the tipping point analysis. We recommend integrating this tipping point analysis as a standard practice in AB-NMA.</p>","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 5","pages":"797-812"},"PeriodicalIF":8.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12527527/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A comparison of combined <i>p</i>-value functions for meta-analysis.","authors":"Leonhard Held, Felix Hofmann, Samuel Pawel","doi":"10.1017/rsm.2025.26","DOIUrl":"10.1017/rsm.2025.26","url":null,"abstract":"<p><p><i>P</i>-value functions are modern statistical tools that unify effect estimation and hypothesis testing and can provide alternative point and interval estimates compared to standard meta-analysis methods, using any of the many <i>p</i>-value combination procedures available (Xie et al., 2011, JASA). We provide a systematic comparison of different combination procedures, both from a theoretical perspective and through simulation. We show that many prominent <i>p</i>-value combination methods (e.g. Fisher's method) are not invariant to the orientation of the underlying one-sided <i>p</i>-values. Only Edgington's method, a lesser-known combination method based on the sum of <i>p</i>-values, is orientation-invariant and still provides confidence intervals not restricted to be symmetric around the point estimate. Adjustments for heterogeneity can also be made and results from a simulation study indicate that Edgington's method can compete with more standard meta-analytic methods.</p>","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"16 5","pages":"758-785"},"PeriodicalIF":8.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12527540/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146103387","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}