Artificial intelligence in occupational therapy documentation: Chatbot vs. Occupational Therapists.

Abstract

Background: Artificial intelligence (AI)-based language models such as ChatGPT show promise in generating medical documentation. However, their effectiveness in occupational therapy (OT) documentation-particularly in terms of perceived quality and empathy-remains underexplored.

Objective: This study aimed to compare the quality and empathy of clinical documentation generated by licensed occupational therapists versus ChatGPT-3.5, using standardized OT case scenarios.

Methods: Fifteen standardized OT cases were used to generate human- and AI-written assessment and plan sections. Five occupational therapists and five patients or caregivers independently evaluated the documentation using 5-point Likert scales across three quality subdomains (completeness, correctness, concordance) and three empathy dimensions (cognitive, affective, behavioral). Inter-rater reliability and correlations between quality and empathy were also analyzed.

Results: Artificial intelligence-generated documentation received significantly higher ratings across all quality and empathy dimensions than human-generated documentation (all p < 0.001). However, human-generated documentation demonstrated stronger correlations between quality and empathy, and higher inter-rater reliability, indicating greater consistency among evaluators. These findings suggest that while AI can produce responses perceived as more complete and empathetic, its outputs may vary more widely in interpretation.

Conclusion: Artificial intelligence-based tools may help reduce documentation burdens for therapists by generating high-quality, empathetic notes. However, human-authored documentation remains more consistent across evaluators. These results underscore the potential and limitations of AI in clinical documentation, highlighting the need for further development to enhance contextual sensitivity, communication coherence, and evaluator reliability. Future research should examine AI performance in real-world OT practice settings.