Treatment Fidelity in a Feasibility Trial of the Aphasia Intervention, Virtual Elaborated Semantic Feature Analysis

Background and Aims

The reliability and validity of an intervention can be improved by checking treatment fidelity (TF). TF methods identify core components of an intervention, check their presence (or absence) and identify threats to fidelity. The Virtual Elaborated Semantic Feature Analysis (VESFA) intervention comprised individual sessions of word-finding treatment and group sessions of conversation practice. All sessions were delivered in the virtual world of EVA Park. This paper describes the TF in the VESFA trial that explored (1) if the treatment was delivered as planned, (2) which components influenced treatment adherence scores and (3) the reliability of the fidelity checklists.

Methods and Procedures

Strategies to improve fidelity were employed in the study design, the delivery of treatment, treatment receipt and treatment enactment. Two fidelity checklists were developed with input from advisors with aphasia to establish the core components of the intervention (individual and group). During the trial, treatment sessions were video-recorded. A sample of 20% of sessions was randomly selected for adherence rating. Seven research students were trained to rate the videos using the fidelity checklists. Inter- and intra-rater reliability was established.

Outcomes and Results

Study design strategies ensured 94% of sessions ran as planned and 75% of participants (12/16) received over 90% (>36/40h) of the intended dose. The average TF across all sessions rated was 81%, demonstrating a high degree of fidelity in the delivery of the VESFA intervention. The fidelity of the individual sessions was lower (78%) than the group elements (84%). The components that most threatened treatment adherence were (1) providing a rationale for the activities and (2) specific feedback for performance. Nevertheless, participants consistently practised target words both in individual sessions and in conversations in the group sessions, demonstrating treatment receipt. Ninety-four percent of participants (14/15) reported the words and phrases practiced in EVA Park were used in real-world conversations, indicating treatment enactment. The fidelity checklists were reliable: Inter-rater reliability was moderate (average Kappa of 0.76) and intra-rater reliability was strong (average Kappa of 0.89).

Conclusions and Implications

A range of TF strategies were embedded within the trial protocol leading to high adherence to the core components of the VESFA intervention. Findings add to the evidence that aphasia therapies can be administered faithfully within the virtual environment of EVA Park.

Trial Registration

The feasibility trial was not registered.

WHAT THIS PAPER ADDS

What is already known on this subject

• Monitoring treatment fidelity improves both internal and external validity. Reports of treatment fidelity from aphasia trials are increasing, but the guidance is not yet applied uniformly.

What this study add to the existing knowledge

• This study demonstrates how treatment fidelity guidance has been applied across a range of fidelity areas to monitor and support a feasibility trial of a novel aphasia intervention. It is a rare reporting of strategies to monitor treatment enactment.

What are the potential or actual clinical implications of this study?

• This study adds to the evidence base for the VESFA intervention, demonstrating that the intervention can be delivered faithfully to the manual. It builds on the evidence base for treatment fidelity monitoring in aphasia, broadening the strategies to improve the validity of interventions.