Let's chat(GPT): Implementation of a ChatGPT-generated social determinants of health activity

Background

Artificial intelligence (AI)- powered chatbots have provided some notable benefits for learners. Educators are beginning to explore their possible utility and find ways to leverage AI in their classrooms.

Objective

This study aimed to evaluate the implementation of ChatGPT-generated social determinants of health (SDOH) activity in a team-based pharmacy education course.

Methods

Instructors asked the software to generate a set of learning objectives, an in-class activity, assessment strategies, and summative assessments for a student's conceptual understanding of SDOH. During a required first-year, team-based pharmacy course at the University of Tennessee Health Science Center, participants (n = 95) completed a ChatGPT-generated, in-class activity on SDOH within groups. The students' views on the quality of the activity were evaluated using five Likert-scale questions. Four of the questions assessed the applicability and usefulness of the assignment, with rankings on a scale of 1–4 (1 = strongly disagree; 4 = strongly agree). The fifth question evaluated the quality of the activity compared to activities generated by the instructor, using a scale of 1–5 (1 = far worse; 5 = far better).

Results

For applicability and usefulness,”94.7 % (n = 90) of students agreed that “This in-class exercise was valuable to my professional development as a pharmacist;” 96.8 % (n = 92) agreed with “It is necessary for pharmacists to understand SDOH;” 94.7 % (n = 90) agreement with the statement, “The quality of this in-class activity was on-par with other in-class activities in the course;” and 90.5 % (n = 86) agreed with “This in-class exercise was just as impactful to my professional development as other in-class activities.” The majority of students (63.2 %; n = 60) selected either “somewhat better” (40 %; n = 38) or far better (23.2 %; n = 22) for, “Regarding quality, I feel the in-class activity was ____ than other in-class activities.”

Conclusions

Most students reported that the ChatGPT-generated activity on social determinants of health was useful, applicable, and somewhat or far better than instructor activities. However, AI can generate incorrect information and potentially hinder student learning of conceptual frameworks; thus, instructors should review all output carefully.