Research Synthesis Methods最新文献

Meta-analysis with JASP, Part I: Classical approaches. 运用JASP进行元分析，第一部分：经典方法。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-11-06 DOI: 10.1017/rsm.2026.10127

František Bartoš, Eric-Jan Wagenmakers, Wolfgang Viechtbauer

引用次数: 0

A little-known perspective on the I-squared statistic. 关于i平方统计的一个鲜为人知的观点。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-11-03 DOI: 10.1017/rsm.2026.10125

Dan Jackson, Julian P T Higgins, José Antonio López-López

引用次数: 0

Between-case incidence rate ratio for single case experimental designs with count outcomes: A Monte Carlo simulation with conditional and marginal models. 具有计数结果的单例实验设计的病例间发生率比：具有条件模型和边际模型的蒙特卡罗模拟。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-10-21 DOI: 10.1017/rsm.2026.10126

Haoran Li, Chendong Li, Wen Luo, Eunkyeng Baek

{"title":"Between-case incidence rate ratio for single case experimental designs with count outcomes: A Monte Carlo simulation with conditional and marginal models.","authors":"Haoran Li, Chendong Li, Wen Luo, Eunkyeng Baek","doi":"10.1017/rsm.2026.10126","DOIUrl":"https://doi.org/10.1017/rsm.2026.10126","url":null,"abstract":"Calculating design-comparable effect sizes in single-case experimental designs (SCEDs) is essential for research synthesis to identify evidence-based practices that integrate findings from both SCEDs and group-based designs. The field has made significant progress in developing additive design-comparable effect sizes in recent years. This study aims to evaluate the statistical properties of a newly developed proportional estimator, the between-case incidence rate ratio (BC-IRR), for SCEDs with count outcomes. Two analytical frameworks are introduced for estimating BC-IRR: generalized linear mixed models (GLMMs) and generalized estimating equations (GEEs). A large-scale Monte Carlo simulation is conducted to assess the bias of the point estimate, the standard error bias, mean squared error, and coverage rate. We also demonstrate the estimation of BC-IRR using GLMMs and GEEs with real data. Based on the simulation and demonstration results, we provide recommendations for applied researchers, discuss limitations, and outline directions for future research.","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":" ","pages":"1-30"},"PeriodicalIF":8.0,"publicationDate":"2026-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149230872","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Evaluating prompting strategies and large language models in systematic literature review screening: Relevance and task-stage classification. 评价系统性文献综述筛选中的提示策略和大语言模型：相关性和任务阶段分类。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-10-19 DOI: 10.1017/rsm.2026.10110

Binglan Han, Anuradha Mathrani, Vajisha Udayangi Wanniarachchi, Teo Susnjak

{"title":"Evaluating prompting strategies and large language models in systematic literature review screening: Relevance and task-stage classification.","authors":"Binglan Han, Anuradha Mathrani, Vajisha Udayangi Wanniarachchi, Teo Susnjak","doi":"10.1017/rsm.2026.10110","DOIUrl":"https://doi.org/10.1017/rsm.2026.10110","url":null,"abstract":"This study quantifies how prompting strategies interact with large language models (LLMs) to automate the screening stage of systematic literature reviews (SLRs). We evaluate six LLMs (GPT-4o, GPT-4o-mini, DeepSeek-Chat-V3, Gemini-2.5-Flash, Claude-3.5-Haiku, and Llama-4-Maverick) under five prompt types (zero-shot, few-shot, chain-of-thought [CoT], CoT-few-shot, and self-reflection) across relevance classification and six Level-2 tasks, using accuracy, precision, recall, and F1. Results show pronounced model-prompt interaction effects within this SLR-automation corpus: zero-shot maximizes recall for high-sensitivity relevance screening; CoT-few-shot yields stronger precision and F1; and self-reflection underperforms due to over-inclusivity and instability across models. GPT-4o and DeepSeek provide robust overall performance, while GPT-4o-mini performs competitively at a substantially lower dollar cost. A cost-performance analysis for relevance classification (per 1,000 abstracts) reveals large absolute differences among model-prompt pairings; GPT-4o-Mini remains low-cost across prompts, and structured prompts (CoT/CoT-few-shot) on GPT-4o-Mini offer attractive F1 at a small incremental cost. The cost-performance analysis also suggests that future screening workflows could incorporate staged, cost-aware procedures: (1) using lower-cost, high-recall configurations for initial screening and (2) reserving higher-cost or more precision-oriented configurations for later triage or uncertain records. These findings highlight LLMs' uneven but promising potential to support screening within corpora similar to the SLR-automation corpus examined in this study. Further validation is needed before generalising these results to broader systematic review screening tasks. By systematically analyzing prompt-model interactions, we provide a comparative benchmark and practical guidance for task-adaptive LLM deployment.","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":" ","pages":"1-40"},"PeriodicalIF":8.0,"publicationDate":"2026-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149222872","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Rethinking time-lag bias in meta-analysis: Broader definitions and conceptual frameworks. 重新思考元分析中的滞后偏差：更广泛的定义和概念框架。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-10-16 DOI: 10.1017/rsm.2026.10121

Xing Xing, Mengli Xiao, Christopher H Schmid, Haitao Chu, Jiayi Tong, Lifeng Lin

{"title":"Rethinking time-lag bias in meta-analysis: Broader definitions and conceptual frameworks.","authors":"Xing Xing, Mengli Xiao, Christopher H Schmid, Haitao Chu, Jiayi Tong, Lifeng Lin","doi":"10.1017/rsm.2026.10121","DOIUrl":"https://doi.org/10.1017/rsm.2026.10121","url":null,"abstract":"Meta-analysis is often viewed as a gold standard for synthesizing available evidence, but it faces challenges as evidence becomes available gradually and studies may be disseminated selectively and at different times. When publication timing is systematically related to study results, meta-analytic conclusions at any given time can be biased; for example, studies with statistically significant or larger effects may be published earlier than those with null or smaller effects. This phenomenon, commonly referred to as time-lag bias, is a key reason why early meta-analytic conclusions may later weaken, reverse, or become more heterogeneous as additional evidence accrues. To address the limitations of existing definitions and detection approaches, which are largely tied to monotone, effect-size-driven dissemination, we broaden the concept of time-lag bias beyond the traditional yet simple notion that \"positive\" studies are published earlier. Specifically, we view time-lag bias more generally as a manifestation of time-dependent evidence availability, whereby the timing of study dissemination can distort meta-analytic conclusions at different stages of evidence accrual. We illustrate this broader perspective through several real-world case studies, including classical time-lag bias, inverse time-lag bias, non-monotone temporal patterns, and anomalous evidence localized within specific time windows. These examples show that time-lag bias can arise through multiple dissemination mechanisms and may exaggerate, attenuate, or even reverse pooled estimates over time. They highlight the importance of explicitly accounting for the temporal dimension of evidence accumulation and motivate the routine use of diagnostic and sensitivity strategies to assess robustness to selective dissemination over time.","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":" ","pages":"1-15"},"PeriodicalIF":8.0,"publicationDate":"2026-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149190421","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Network meta-analysis and diffusion. 网络荟萃分析和扩散。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-10-15 DOI: 10.1017/rsm.2026.10123

Gerta Rücker, Annabel Louisa Davies, Guido Schwarzer

引用次数: 0

MechaScreener: Large language model-based automated screening for systematic reviews and research. MechaScreener：用于系统评论和研究的基于大型语言模型的自动筛选。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-10-15 DOI: 10.1017/rsm.2026.10124

Connor Forbes, Matt Carter, Carly Hudson, Paul Glasziou, Justin Clark

{"title":"MechaScreener: Large language model-based automated screening for systematic reviews and research.","authors":"Connor Forbes, Matt Carter, Carly Hudson, Paul Glasziou, Justin Clark","doi":"10.1017/rsm.2026.10124","DOIUrl":"https://doi.org/10.1017/rsm.2026.10124","url":null,"abstract":"Systematic reviews are the gold standard for evidence synthesis, but the manual title and abstract screening of thousands of references creates a severe bottleneck. Existing automated tools have historically struggled to achieve the near-perfect recall (sensitivity) required for reliable reviews. We developed MechaScreener as a 'zero-shot' automated screening tool that utilises a large language model to rank article relevance. The tool requires no initial training data or manual pre-screening, as MechaScreener directly applies user-provided question elements (PICO) or inclusion/exclusion criteria to assign an ordinal relevance score (1-5) to each reference. We evaluated the tool in two phases: a development phase using five reference libraries to optimise prompts, and an independent evaluation phase using 10 diverse Cochrane review libraries (comprising both randomised controlled trials [RCTs] and non-RCTs) containing over 58,000 references. In the evaluation dataset, MechaScreener achieved a perfect recall of 1.00 (100%) in all 10 reviews, ensuring that no relevant articles were missed (pooled record-level 95% CI: 0.98-1.00). Concurrently, it achieved an overall macro-average specificity of 0.61 (61%, cluster bootstrap 95% CI: 0.46-0.75; pooled record-level specificity 0.59, 95% CI: 0.59-0.60). Specificity varied from 0.21 in broad public health topics to 0.91 in precise pharmacological interventions, reflecting the tool's built-in conservatism when evaluating ambiguous abstracts. By safely eliminating over 60% of irrelevant literature during the initial screening phase without compromising recall, MechaScreener functions as a highly reliable but low-effort 'first-pass' filter, allowing researchers to substantially reduce manual workloads and reallocate resources towards full-text review and data extraction.","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":" ","pages":"1-10"},"PeriodicalIF":8.0,"publicationDate":"2026-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149178358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Handling within-study variability in moderators and dependence in correlation coefficients: Meta-analytic structural equation modeling. 处理研究内调节因子的可变性和相关系数的依赖性：元分析结构方程模型。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-10-15 DOI: 10.1017/rsm.2026.10113

Jihyun Lee, S Natasha Beretvas

{"title":"Handling within-study variability in moderators and dependence in correlation coefficients: Meta-analytic structural equation modeling.","authors":"Jihyun Lee, S Natasha Beretvas","doi":"10.1017/rsm.2026.10113","DOIUrl":"https://doi.org/10.1017/rsm.2026.10113","url":null,"abstract":"Meta-analytic structural equation modeling (MASEM) is a flexible modeling framework that combines the strengths of meta-analysis and structural equation modeling. Despite the increasing popularity of the MASEM technique, its usage and scope are limited to relatively simple meta-analytic data structures and moderator hypotheses. Within-study variability in moderator variables and dependence in primary studies' correlation estimates tend to be handled using ad hoc methods. In this study, we demonstrate use of multilevel multivariate modeling with robust variance estimation (RVE) for handling within-study dependence when pooling studies' correlation estimates and we propose how to incorporate categorical moderator values directly into the structural equation model for moderators with values varying within studies. We outline the proposed approach using three stages: (a) specification of the structural equation model using moderator variable categories, (b) pooling correlation matrix elements using the three-level multivariate model with RVE, and (c) testing the moderator hypotheses through either model comparison or direct estimation of structural equation model's path differences. We illustrate the approach using three examples to demonstrate its flexibility across different MASEM models. This framework broadens the scope of moderator hypotheses that can be tested using MASEM and enhances the methodological rigor for handling dependencies in studies' correlation estimates for MASEM.","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":" ","pages":"1-28"},"PeriodicalIF":8.0,"publicationDate":"2026-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149178378","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Artificial intelligence tools for evidence synthesis in the biomedical field: A living rapid review and evidence map. 生物医学领域证据合成的人工智能工具：一个活生生的快速回顾和证据地图。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-10-08 DOI: 10.1017/rsm.2026.10118

Gaelen P Adam, Melinda Davies, Eduardo Caputo, Htun Ja Mai, Erin Coppola, Haley Holmer, Edi Kuhn, Ethan M Balk, Thomas A Trikalinos, Ilya Ivlev, Holly R Wethington

{"title":"Artificial intelligence tools for evidence synthesis in the biomedical field: A living rapid review and evidence map.","authors":"Gaelen P Adam, Melinda Davies, Eduardo Caputo, Htun Ja Mai, Erin Coppola, Haley Holmer, Edi Kuhn, Ethan M Balk, Thomas A Trikalinos, Ilya Ivlev, Holly R Wethington","doi":"10.1017/rsm.2026.10118","DOIUrl":"https://doi.org/10.1017/rsm.2026.10118","url":null,"abstract":"Evidence synthesis (ES) is costly and time-intensive. Artificial intelligence (AI) tools are increasingly used to automate or semi-automate ES tasks. We conducted a living rapid review of AI tools that automate or semi-automate ES tasks. We searched for published articles and preprints through MEDLINE, Embase, and the Association for Computing Machinery Digital Library from January 1, 2021 through October 15, 2025, supplemented by hand-searching. We included existing reviews to capture evaluations before 2021. Screening was conducted in duplicate at the title/abstract level, with single full-text screening and data extraction verified by a second reviewer. We included 134 studies. Performance varied by task and evaluation approach. Search automation tools showed poor recall (median 17%), although human-in-the-loop approaches improved performance. Tools for identifying randomized controlled trials demonstrated high median recall (98%) and precision (92%). Semi-automated abstract screening tools using active learning reduced screening burden by approximately 50% while maintaining 95% recall. Large language models demonstrated variable accuracy for abstract screening (median 88% recall), full-text screening (median 96% recall), and data extraction (median 63% accuracy). Risk of bias (RoB) assessment tools showed moderate agreement with human reviewers for the Cochrane RoB tool version 1 (median 71%) but performed less well with other RoB tools. AI tools can meaningfully reduce burden in study design identification and abstract screening but perform inconsistently across other evidence synthesis tasks. Tools for search automation, data extraction, and RoB assessment require further development. No existing tool can currently replace human expertise.","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":" ","pages":"1-25"},"PeriodicalIF":8.0,"publicationDate":"2026-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149134400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Meta-analysis reconceived from a finite population sampling perspective. 从有限总体抽样的角度重新构思的荟萃分析。

IF 8 2区生物学

Research Synthesis Methods Pub Date : 2026-09-24 DOI: 10.1017/rsm.2026.10115

Matthew Forte, Elizabeth Tipton

{"title":"Meta-analysis reconceived from a finite population sampling perspective.","authors":"Matthew Forte, Elizabeth Tipton","doi":"10.1017/rsm.2026.10115","DOIUrl":"https://doi.org/10.1017/rsm.2026.10115","url":null,"abstract":"In current meta-analytic methods, researchers must choose which model to use-the common effect (CE), fixed effects (FES), or random effects (RE) model. The CE and FES models allow for estimation of the person-average effect size while conditioning on the studies included in the analysis, while the RE model allows for generalizations to a broader population of studies, yet may estimate the study-average effect rather than the person-average effect. These meta-analysis methods (CE, FES, and RE) do not always allow for both estimation of the person-average effect and generalizations beyond the studies included in the analysis. In this article, we consider situations in which the person-average and study-average effect size can differ. Then, we propose a new framework that models directly how studies are sampled into a meta-analysis. We consider three different sampling designs-simple random sampling (SRS), stratified SRS, and cluster sampling-and investigate how a meta-analysis could be conceived under these design frameworks. Through this new framework, we introduce assumptions, estimators, and weights that can be used to estimate person-average effects with inference that is no longer conditional on the observed studies. We then illustrate how these new models and estimators differ using examples based upon data in the What Works Clearinghouse and a pre-existing systematic review. We conclude with a discussion of the implications of these differences in the context of the proposed sampling estimators, as well as suggestions for using these models and future research.","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":" ","pages":"1-26"},"PeriodicalIF":8.0,"publicationDate":"2026-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149035672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0