ReviewGenie: a novel automated system for systematic reviews-an exploratory study in speech and language disorders.

Abstract

Background: Systematic reviews (SRs) are a cornerstone in providing high-quality evidence that guides policy and practice across various disciplines. Despite their critical role, SRs require substantial financial investment and are constrained by time-consuming manual processes. Existing solutions primarily focus on semi-automating the title and abstract screening stages, yet these approaches still face limitations in terms of efficiency and practicality. The SR process comprises several stages beyond abstract screening, each of which is resource-intensive. To overcome these challenges, this paper introduces ReviewGenie, a novel system that automates SR stages up to and including abstract screening, utilizing artificial intelligence.

Method: The SR process involves eight key stages, beginning with the definition of search keywords and the selection of target databases, and culminating in full screening. While the initial and final stages require human expertise, the intermediate stages can be automated. ReviewGenie automates all intermediary stages, including database searching, data retrieval, cleaning, deduplication, filtering, and abstract screening. The system is domain-agnostic, as evidenced by a case study focused on databases related to speech and language disorders.

Results: ReviewGenie significantly reduces the workload across various stages of the SR process, delivering notable time and cost savings while enhancing efficiency and accuracy. In the case study, where the article-fetching stage involved tens of thousands of publications, ReviewGenie achieved a 2.62% improvement in duplicate detection in less than a second, compared to the 1 to 3 h typically required for manual deduplication of 100 records. This process included cleaning abstracts before removing duplicates. Additionally, ReviewGenie reduced the number of articles from 28,674 to 3520 using an automatic filtering approach executed in seconds. This substantial reduction underscores the effectiveness of our automated method in preparing datasets for the abstract screening stage. Moreover, the artificial intelligence-driven abstract screening method resulted in cost savings exceeding $6230 compared to manual methods.

Conclusions: ReviewGenie represents a significant advancement in reducing the burden on researchers conducting comprehensive systematic reviews. By automating intermediate stages, ReviewGenie enhances efficiency, accuracy, and cost-effectiveness, establishing itself as an indispensable tool for SRs across various disciplines.