Exploring ChatGPT-4o-generated reflections: Alignment with professional standards in diagnostic radiography - A pilot experiment.

Abstract

Introduction/background: Artificial intelligence (AI) tools such as ChatGPT-4o are increasingly being explored in education. This study examined the potential of ChatGPT-4o to support reflective practice in medical radiation science (MRS) education. The focus was on the quality of AI-generated reflections in terms of alignment with professional standards, depth, clarity, and practical relevance.

Methods: Four clinical scenarios representing third-year diagnostic radiography placements were used as prompts. ChatGPT-4o generated reflective responses, which were assessed by three reviewers. Reflections were evaluated against the Medical Radiation Practice Board of Australia's professional capability domains and the National Safety and Quality Health Service Standards. Review criteria included clarity, depth, authenticity, and practical relevance. Inter-rater reliability was analysed using intraclass correlation coefficients (ICC) and the Friedman test.

Results: Scenario 3 achieved the highest inter-rater reliability (ICC: moderate to excellent; p = 0.022). Scenario 2 showed the lowest reliability (ICC: poor to fair; p = 0.060). Reflections were consistently well-structured and clear, but often lacked emotional depth, contextual awareness, and person-centered insights. Qualitative feedback identified limitations in empathetic reflection and critical self-awareness.

Discussion: ChatGPT-4o can produce structured reflective responses aligned with professional frameworks. However, its lack of emotional and contextual depth limits its ability to replace authentic reflective practice. Reviewer agreement varied depending on scenario complexity and emotional content.

Conclusion: AI tools such as ChatGPT-4o can assist in structuring reflections in MRS education but should complement, not replace, human-guided reflective learning. Hybrid models combining AI and educator input may enhance both efficiency and authenticity.