The USE Test in History and Its Validity: Experts of Regional Subject-Specific Committees Speculating on Free-Response Items

Oleg Fedorov, Candidate of Sciences in History, Associate Professor, Director of the Siberian Institute of Management, Russian Presidential Academy of National Economy and Public Administration (RANEPA). Address: 6 Nizhegorodskaya Str., 630102 Novosibirsk, Russian Federation. E-mail: fedorov-od@ranepa.ru (corresponding author) Ksenia Verinchuk, History Teacher, European Gymnasium. Address: 28 Sokolnicheskiy Val Str., 107113 Moscow, Russian Federation. E-mail: k.verinchuk@gmail.com The Unified State Exam (USE) in Russia is both an achievement and admission test, yet its validity has not been looked into on a large scale. The evolution of USE tests is distinctly marked by a growing number of constructed-response items, which might be affecting the validity of test results in many ways. In-depth semi-structured interviews with 36 USE experts in history allow identifying three major threats to USE validity: assessment criteria for items 24 and 25, item content, and expert bias. Interview transcripts were analyzed using content analysis, the results of which are presented along with recommendations on how to further improve the processes of item design and evaluation.