Dependability of individualized Direct Behavior Rating Multi-Item Scales (DBR-MIS) for academic enablers

Abstract

The present study examined the dependability of three newly developed direct behavior rating multi-item scales (DBR-MIS) of academic enablers (i.e., academic engagement, interpersonal skills, and study skills). Twenty-two K–5 teachers completed all three 5-item DBR-MIS daily for 1 week for one student in their class. Teachers' ratings on each item during the first occasion were used to create individualized DBR scales with 1–4 items. Items with the lowest ratings (indicating least frequent academic enablers) were included first and subsequent items were added in ascending order. Dependability of both full DBR-MIS and individualized DBR scales was evaluated using generalizability theory. Results indicated that the full DBR-MIS demonstrated high dependability and required only 1–4 assessment occasions (i.e., < 10 as the criterion) to inform absolute decision-making for progress monitoring. The three- and four-item individualized DBR-MIS demonstrated comparable dependability to their respective full five-item DBR-MIS. Dependability estimates of individualized scales in general were higher than standard D study-derived estimates with the same number of items (i.e., dependability estimates obtained by manipulating the number of items from the full standard scales modeled in D studies). Results support continued investigation of the DBR-MIS as a viable progress monitoring tool for school-based applications. Further research and implications for practice were discussed.