Knowledge and Attitudes toward an Artificial Intelligence-Based Fidelity Measurement in Cognitive Behavioral Therapy Supervision

Purpose. To capitalize on investments in evidence-based practice implementation, technology is needed to scale up fidelity assessment. Stakeholder feedback may facilitate adoption of such tools. This study examined mental health providers’ perceptions of an automated fidelity-rating tool for Cognitive Behavioral Therapy (CBT). Methods. We administered online surveys before and after focus groups that included therapists (n=18) and clinical leadership (n=12) from community mental health programs implementing CBT. Discussions centered on supervision practices followed by feedback about the fidelity tool. Transcripts were analyzed using a grounded theory approach.Results. Initial perceptions of acceptability, appropriateness, and feasibility of automated CBT fidelity tools were positive and increased after introduction of the tool, including significant increases in ratings of acceptability and appropriateness. Standard supervision was described as collaboratively guided, either scheduled or spontaneous, and focused on clinical content, self-care, and documentation. Participants highlighted the tool’s utility for supervision, training, and professional growth, but questioned its ability to rate skills related to rapport, cultural diversity, and non-verbal communication. Concerns were raised about client and therapist privacy, and the impact of low scores on therapist confidence. Desired features included labeling of interventions used and transparency about how fidelity scores related to specific point in the session. Opportunities for asynchronous, remote, and targeted supervision were of particular value. Conclusions. Automated fidelity assessment may present an opportunity for an acceptable, appropriate, and feasible approach to large-scale EBP implementation. Continued partnership with community stakeholders will be key for designing such tools in a manner that increases the likelihood of uptake.