Assessing Interobserver Agreement (IOA) with Procedural Integrity: Evaluation of Training Methods among Classroom Instructors

Abstract

Abstract This study evaluated training methods to improve procedural integrity of classroom instructorswhen assessing interobserver agreement (IOA) with their students. Compared to baseline conditions in a multiple baseline across settings design, training that included a memo-task analysis review did not improve procedural integrity. Subsequent intervention consisting of single-session group administered behavioral skills training (BST: instruction, demonstration, role-playing, performance feedback) increased and stabilized procedural integrity of all instructors. These findings support previous training research concerned with procedural integrity of care providers and a competency-based approach towards improving IOA assessment.