Impact of artificial intelligence on the academic performance and test anxiety of pharmacy students in objective structured clinical examination: a randomized controlled trial.

Abstract

Background: The rapid advancement of generative artificial intelligence (AI) in recent years has led to its increased application across various fields including education. One area where AI can significantly impact is clinical education, particularly in the preparation and execution of objective structured clinical examinations (OSCEs).

Aim: This study aimed to evaluate the impact of AI-generated study material and feedback on the academic performance and level of anxiety of pharmacy students in formative OSCE.

Method: This was a 4-week (June-July 2024) randomized controlled study. Students of 6th semester PharmD program were randomized into either an intervention or control group. The intervention group received intervention which comprised a comprehensive training session on how to use AI tools (ChatGPT, Gemini and Perplexity) for generating study materials with personalized feedback, in addition to usual OSCE instructions. The control group only received the usual OSCE instructions. In addition, all students completed the test anxiety inventory (TAI) questionnaire before the OSCE.

Results: Eighty-eight (40 male, 48 female) out of 92 (96%) students attended the OSCE and completed the TAI questionnaire. Each group had 44 (50%) students. The mean OSCE mark was 13.26 (± 5.05) out of 30. No significant difference was found between the intervention [12.98 (± 5.15)] and control [13.54 (± 5.00)] groups regarding mean OSCE marks (p = 0.550). Similarly, no significant difference was found between the groups regarding the total TAI score (p = 0.917).

Conclusion: While the use of AI tools did not improve the academic performance of students or reduce test-related anxiety, they did not negatively impact these outcomes either. Future research should investigate the long-term effects of AI-based interventions on educational outcomes.