Preliminary investigation of an artificial intelligence-based cognitive behavioral therapy training tool.

Abstract

We developed an asynchronous online cognitive behavioral therapy (CBT) training tool that provides artificial intelligence- (AI-) enabled feedback to learners across eight CBT skills. We sought to evaluate the technical reliability and to ascertain how practitioners would use the tool to inform product iteration and future deployment. We conducted a single-arm 2-week field trial among behavioral health practitioners who treat outpatients with psychosis. Practitioners (N = 21) were invited to use the AI-enabled CBT training tool over a 2-week (15 days, inclusive) period. To enable naturalistic observation, no adjustments were made to their workloads nor were prescriptions on use provided. We conducted daily assessments and collected backend analytics for all users. At end point, we assessed acceptability, appropriateness, feasibility of implementation, perceived usability, satisfaction, and perceived impact of training. We observed four types of technical issues: broken links, intermittent issues receiving AI-enabled feedback, video replay errors, and an HTML error. Participants averaged 6.57 logins over the 2 weeks, with more than half engaging daily. Most participants (44.7%) engaged for < 30-min increments. Usability scores exceeded industry standard and satisfaction scores indicated good promotion of the tool. All participants endorsed high feasibility, acceptability, and appropriateness. Twelve participants (57%) used the AI-enabled feedback feature; those who did tended to report improved satisfaction, feasibility, and perceived impact of the training. The training tool was used by practitioners in a routine care setting, met or exceeded conventional implementation benchmarks, and may support skill improvement; however, data suggest that practitioners may need support or accountability to fully leverage the training tool. (PsycInfo Database Record (c) 2025 APA, all rights reserved).