Evaluating Compliance of Randomized Controlled Trial Abstracts in Plastic Surgery Journals with CONSORT Guidelines Using GPT-4 AI.

Abstract

Background: The quality of reporting in randomized controlled trials (RCTs) is crucial for accurate interpretation and synthesis of evidence. The Consolidated Standards of Reporting Trials (CONSORT) guidelines provide a standardized framework for reporting RCT abstracts. This study aimed to evaluate the adherence of RCT abstracts published in three major plastic surgery journals to the CONSORT tool guideline for reporting abstracts, utilizing Generative Pre-trained Transformer 4 artificial intelligence (GPT-4 AI) technology.

Methods: Abstracts of RCTs published between 2010 and 2023 were collected. The GPT-4 AI model was utilized to assess the abstracts based on the CONSORT criteria. Descriptive statistics were used to report the compliance scores and identify areas where abstracts lacked compliance.

Results: Of the initially identified 500 abstracts, a total of 371 RCT abstracts met the inclusion criteria and were analyzed. The mean CONSORT score was 10.05 (±2.22), with a median score of 10.72. Specific areas where abstracts lacked compliance included trial design (39.6%), participant details (28.8%), intervention descriptions (15.6%), randomization process (25.3%), and the number of participants analyzed (33.4%). Trial registration (18.3%) and funding information (15.1%) were also frequently missing.

Conclusions: Our study's innovative use of the GPT-4 AI model for analysis demonstrated the potential of AI technology in streamlining and enhancing the evaluation of research compliance. We advocate for heightened awareness and more rigorous application of CONSORT guidelines among authors, reviewers, and journal editors. Emphasizing the role of AI technology in the evaluative process can further improve the reporting quality of future RCTs in plastic surgery, contributing to more reliable and transparent research in the field.