Can the Oral Proficiency Interview - Computer (ACTFL OPIc) be used instead of the Oral Proficiency Interview (ACTFL OPI)? An aligned rank transform (ART) analysis

This study investigated the differences between the ACTFL Oral Proficiency Interview (OPI) and the ACTFL Oral Proficiency Interview - Computer (OPIc) among Spanish learners at a U.S. university. Participants (N = 154) were randomly assigned to take both tests in a counterbalanced order to mitigate test order effects. Data were analyzed using an aligned rank transform (ART) analysis, focusing on variables such as gender, age, language courses, missionary experience, and self-assessed Spanish ability. Results showed a strong correlation between ACTFL OPI and ACTFL OPIc ratings (τ = 0.79, p < .001), though ACTFL OPIc ratings were slightly higher on average. No significant order effects were found, indicating the order of test administration did not influence ratings. The reliability of both tests was confirmed, and no significant biases were detected. The findings suggest that both ACTFL OPI and ACTFL OPIc are effective, holistic measures of Spanish oral proficiency, with ACTFL OPIc offering a slight advantage in rating outcomes. Pedagogically, this suggests flexibility in test choice without compromising assessment integrity.