Adaptive assessment in certification procedures for students and graduates

Abstract

The competency-based approach, which today serves as the basis for the design and implementation of educational programs for the training of professional specialists, provides a certain set of competencies formed by students and graduates (universal, general professional, professional) as learning outcomes. It is possible to assess competencies as latent characteristics only in activity. This latent characteristic, which is the reason why students and graduates are able to perform professional tasks, is the goal of measurement. But in practice, observable assessments of abilities or skills for performing quasi-professional tasks are obtained, according to which conclusions about the level of formation of latent competencies are made. In turn, the implementation of the competency-based approach involves a change in approaches to assess learning outcomes, since it is impossible to talk about obtaining objective (reliable), comparable (valid) and reliable information about achieved learning outcomes and the level of competency formation. This gives rise to the need to refer to a mixed (biparadigm) methodology of educational measurements, on the one hand, and a special mathematical apparatus of the modern test theory (Item Response Theory — IRT), designed to assess the latent parameters of the subjects and the parameters of the tasks of the assessment tools, on the other hand. The proposed approaches to the organization and conduct of certification procedures for students and graduates provide objective, comparable and reasonable results. Unlike traditional formats for conducting the state exam for the final certification of graduates, adaptive assessment is attractive from the point of view of obtaining more accurate estimates of examinees’ parameters (level of preparedness or level of competence formation) due to fewer tasks and creating a “success situation” for each student in the measurement process through machine algorithms selection of such tasks that he will be able to perform. IRT algorithms based, for example, on the maximum likelihood method, make it possible to implement the humanistic ideas of control and evaluation activities. To implement any of the described adaptive approaches in the intermediate or final certification, the following conditions must be met: availability of a bank of calibrated tasks with stable characteristics (difficulties, differentiating ability) or algorithms for their cloning; availability of computer programs or software and instrumental environment (service, platform) that use one or more selected IRT models that help achieve the highest possible measurement accuracy when assessing the level of preparedness or the formation of student competencies; availability of specifications for assessment tools that ensure the meaningful validity of the measurement results. From a didactic point of view, the latter condition is especially important, since in order to obtain objective and comparable results during certification, it is necessary to take into account the content elements of the educational program, the verification of which is planned in the specification of the assessment tools.