Evaluation of Language and Teaching Skill Domains for International Teaching Assistants: An Approach based on Invariant Measurement

Abstract

ABSTRACT Drawing on the framework of invariant measurement from Rasch measurement theory, the purpose of this study is to psychometrically evaluate the 20 language and teaching skill domains of the International Teaching Assistant (ITA) Test using the many-facet Rasch model and to empirically explore performance differences between females and males in these domains through bias analysis. The data came from the test scores of 110 prospective ITAs on the ITA Test at a large university. Three facets (examinee, rater, and domain) were Rasch-calibrated in FACETS. Despite some misfits, overall, the data fit the model reasonably well, confirming invariant measurement and the feasibility of assessing the language and teaching skill domains concurrently to produce a single score in ITA assessment. The results also indicated that overall language skills were more difficult than teaching skills. Grammar and pronunciation skills were found to be the most difficult domains, whereas the aural comprehension skill was found to be the easiest domain. A bias analysis revealed significant differences in the four domains between the two gender groups, calling for further research for examining potential gender biases in ITA assessment.