The Reflective Fostering Programme fidelity rating scale: development and inter-rater reliability

Purpose The purpose of this study is to describe the development of the 14-item reflective fostering fidelity rating (RFFR), an observational rating system to evaluate model fidelity of group facilitators in the Reflective Fostering Programme (RFP), a mentalisation-based psychoeducation programme to support foster carers. The authors assess usability, dimensionality, inter-rater reliability and discriminative ability of the RFFR. Design/methodology/approach Eighty video clip extracts documenting 20 RFP sessions were independently rated by four raters using the RFFR. The dimensionality of the RFFR was assessed using principal components analysis. Inter-rater agreement was assessed using the intra-class correlation coefficient. Findings The proportion of missing ratings was low at 2.8%. A single principal component summarised over 90% of the variation in ratings for each rater. The inter-rater reliability of individual item ratings was poor-to-moderate, but a summary score had acceptable inter-rater reliability. The authors present evidence that the RFFR can distinguish RFP sessions that differ in treatment fidelity. Originality/value To the best of the authors’ knowledge, this is the first investigation and report of the RFFR’s validity in assessing the programme fidelity of the RFP. The paper concludes that the RFFR is an appropriate rating measure for treatment fidelity of the RFP and useful for the purposes of both quality control and supervision.