Position Statement on Artificial Intelligence (AI) Use in Evidence Synthesis Across Cochrane, the Campbell Collaboration, JBI, and the Collaboration for Environmental Evidence 2025

Abstract

Evidence syntheses, including systematic reviews, are a type of research that uses systematic, replicable methods to evaluate all available evidence on a specific question. They are built on the principles of research integrity, including rigor, transparency, and reproducibility. There is wide recognition that artificial intelligence (AI) and automation have the potential to transform the way we produce evidence syntheses, making the process significantly more efficient. However, this technology is potentially disruptive, characterized by opaque decision-making and black-box predictions, susceptible to overfitting, potentially embedded with algorithmic biases, and at risk of fabricated outputs and hallucinations. To safeguard evidence synthesis as the cornerstone of trusted, evidence-informed decision making, Cochrane, the Campbell Collaboration, JBI and the Collaboration for Environmental Evidence (CEE), have come together to collaborate on a responsible and pragmatic approach to AI use in evidence synthesis.

By AI, we mean different types of automation, as described within the Responsible use of AI in evidence SynthEsis recommendations (RAISE) (Thomas et al. 2025a), specifically, “advanced technologies that enable machines to do highly complex tasks effectively – which would require intelligence if a person were to perform them.” This ranges from general automation applications, such as rule-based or trained machine learning algorithms, to more recent large language models and generative AI approaches.

Incorporating AI in evidence synthesis comes with challenges as well as opportunities. While it is clear we need to make better use of AI for evidence synthesis to become more timely, affordable, and sustainable, we must also acknowledge the environmental and social costs associated with some forms of AI, particularly large-scale language models. There are risks that misuse could erode methodological standards by exacerbating existing biases and reducing reliability (Hanna et al. 2025; Siemens et al. 2025). These concerns are particularly relevant as current AI developments are often driven by commercial interests and, as such, are often opaque regarding limitations and lacking appropriate validation and evaluation. Overall, this undermines the reliability and replicability of AI-driven outputs.

To this end, Cochrane, the Campbell Collaboration, JBI, and the CEE have come together to form a joint AI Methods Group (AI Methods Group 2025). The group officially supports the aims of RAISE (Thomas et al. 2025a), which states that we need to work together to ensure AI does not compromise the principles of research integrity on which evidence synthesis was built. RAISE offers tailored recommendations for roles across the evidence synthesis ecosystem, from evidence synthesists to methodologists, from AI development teams to organizations or publishers involved in evidence synthesis. It is the first step to clarify everyone's responsibilities to ensure safe and responsible AI use. While each of our organizations has different resources and infrastructure, and therefore our approach to implementing the RAISE recommendations may differ, our aim is to align with best practice principles and share lessons learned on effective approaches.

RAISE, and our position on AI use for those who author evidence synthesis with our organizations (Table 1), will continue to evolve as the field and evidence base evolves. We recommend that the most up-to-date version of RAISE (Thomas et al. 2025a) and this published position statement should always be used.

A key area for further guidance is how to approach justifying the use of AI in the specific synthesis. During protocol development, evidence synthesists need to consider many potential trade-offs, often related to balancing capacity and resource availability against questions of urgency, relevance, and scope. It can be helpful to consider the use of AI as an additional trade-off decision you need to make. As part of the decision, evidence synthesists should consider the context of their synthesis (e.g., who will use it and what it is for), what the risk tolerance might be for errors affecting the findings or conclusions of the synthesis, and what potential risk mitigation strategies are available.

In some contexts, the use of AI may enhance the quality of evidence synthesis and help address some of the trade-offs inherent to accelerated timelines. Cochrane's Rapid Review Methods Group has a separate position statement on this (currently under editorial consideration). For example, abstract screening in rapid reviews is often conducted by a single review author, which carries an estimated 13% risk of falsely excluding a relevant study (Gartlehner et al. 2020). Using AI as a second “reviewer” could help reduce this risk.

Evaluation and validation studies are critical to determine whether an AI system or tool performs to an adequate level for evidence syntheses in general, as well as for the specific topics addressed in a given synthesis. Although AI tool developers are responsible for assessing their tools and systems, evaluations by independent methodologists who do not have a conflict of interest in their performance should also be considered whenever available. Clear terms and conditions are vital for ensuring evidence synthesists can adhere to ethical, legal, and regulatory standards. Information about the system and the scope and domains of the data used for training, testing, and validation will help evidence synthesists understand the strengths and limitations, and how well it will apply to their synthesis. For example, the more similar the scope and domains are to your synthesis, the more confidence you could have in using it. Biases may include using English-only or open-access-only data in the training and testing stages, which may or may not be important to the synthesis context. Alternatively, the appropriately justified use of an AI system or tool in another similar-scoped evidence synthesis could form the basis of your justification to use it. Depending on how certain an evidence synthesist is about an AI system or tool in their synthesis, they may need to pilot (or calibrate) the AI system or tool to validate its performance within their evidence synthesis, to ensure its use will not undermine the trustworthiness or reliability of the synthesis or its conclusions. This requires upfront investment from the author team in terms of time, effort, and skill, so a decision will need to be made as to whether the effort is worth the potential gain.

This process to consider the trade-off is the foundation for ensuring human oversight in the decision and application of AI in evidence synthesis. Work is under way to develop a framework that guides evidence synthesists through these considerations, so decisions can be made transparently and consistently.

We understand that this is a new frontier for evidence synthesists to navigate. How do you know when it is appropriate and safe to use AI in your synthesis? In addition to the recommendations for the different roles in the evidence synthesis ecosystem, RAISE also includes guidance for implementing the recommendations in practice, covering building and validation of AI systems and tools, conducting evaluations to build a cumulative evidence base for AI use, performance metrics for AI evaluations, the current status of available AI for evidence synthesis, selecting and using AI in your evidence synthesis, and ethical, legal and regulatory considerations when using AI (Thomas et al. 2025a, 2025b, 2025c).

Ultimately, evidence synthesists are responsible for their work, how they use AI, and any implications, including social and environmental impacts. Our organizations and the joint Methods Group are committed to improving AI literacy for our authors and editors to help make these decisions around AI use. We are aligning our work with developments happening across the ecosystem, including the Evidence Synthesis Infrastructure Collaborative (ESIC 2025), the RAISE initiative (Thomas et al. 2025a), and the Digital Evidence Synthesis Tool INnovation for Yielding Improvements in Climate and Health (DESTINY) project (DESTINY AI-Powered Living Evidence for Climate and& Health 2025), among others. This is to ensure that the members of Cochrane, the Campbell Collaboration, JBI, and CEE, as well as the wider field, have the resources and guidance they need to use AI responsibly, efficiently, and equitably within their evidence synthesis.

This article has been published simultaneously with Cochrane Database of Systematic Reviews, Campbell Systematic Reviews, JBI Evidence Synthesis, and Environmental Evidence. The articles are identical except for minor stylistic and spelling differences in keeping with each journal's style. Any of these journals can be used when citing this article.

All authors are leaders in the joint AI Methods Group with Cochrane, the Campbell Collaboration, JBI, and the Collaboration for Environmental Evidence. Ella Flemyng and Anna Noel-Storr are employed by Cochrane. Zoe Jordan and Kylie Porritt are employed by JBI. Biljana Macura and Matthew Grainger are associated with the Collaboration for Environmental Evidence. Gerald Gartlehner is associated with Cochrane Austria. Jan Minx and Angelika Eisele-Metzger are associated with Cochrane Germany. Jan Minx is associated with Campbell Climate Solutions. Paweł Jemioło is associated with Cochrane Poland. James Thomas, Ella Flemyng, Anna Noel-Storr, Zoe Jordan, Paweł Jemioło, Biljana Macura, Jan Minx, Joerg J. Meerpohl, and Gerald Gartlehner are authors of the Responsible use of AI in evidence SynthEsis recommendations. No authors listed were involved in the editorial process or decisions for this editorial.

This editorial was commissioned based on a co-publication with JBI Evidence Synthesis, Campbell Systematic Reviews, Cochrane Database of Systematic Reviews, and Environmental Evidence. This position statement was fully peer reviewed by leaders of Cochrane, the Campbell Collaboration, JBI, and the Collaboration for Environmental Evidence as part of the endorsement process.