Artificial intelligence versus human analysis: Interpreting data in elderly fat reduction study

Background

Artificial intelligence (AI) has emerged as a transformative tool in scientific research, playing an increasingly significant role in data analysis and interpretation. This study aimed to evaluate the efficiency and accuracy of AI-driven interpretations of health-related data in comparison to those provided by human experts.

Methods

The analysis utilized a paid version of Chat GPT-4 (AI) to interpret study results, relying solely on table titles and data extracted from the authors' previously published manuscript. The dataset encompassed body composition and health parameters within the context of a dietary intervention. Data from a prior publication by the authors were referenced, as detailed in the methods section. The evaluation focused on comparing word count and descriptive content across interpretations of three tables from the original manuscript.

Results

The human expert's data interpretation was succinct, comprising 160 words, while AI-generated descriptions extended to 426 words. Similarly, the AI provided a more verbose analysis of the pre/post-intervention parameter significance, with 374 words compared to the human's 108 words. Group interactions were described in 44 words by the human expert and 486 words by the AI. Notably, the AI's analysis was accurate, though more detailed.

Conclusions

Chat GPT-4 necessitates precise table titles and well-defined data inputs to generate comprehensive analyses, as it does not autonomously account for all parameters within the dataset. While the AI demonstrated factual accuracy and valid conclusions, its descriptions lacked precision, underscoring the importance of human oversight in ensuring interpretative clarity and relevance.