Computerised Testlet Instrument to Assess Students’ Conceptual Understanding of Chemistry on the Topic of Stoichiometry: Psychometric Analysis

Abstract

Measuring students’ conceptual understanding is a challenge for teachers. Teachers usually use open-ended instruments to measure students’ conceptual understanding in depth. However, this instrument is considered highly subjective in the assessment and requires a relatively long time to check the answers. Computerised Testlet instruments developed by combining the advantages of open-ended and closed-ended instruments are deemed the solution to these problems. This study, therefore, aims to analyse the quality of the Computerised Testlet instrument psychometrically. The Rasch analysis model with Winsteps software was used in this study. A total of 10 Testlet items with three supporting questions were developed on the topic of stoichiometry. A total of 413 students (N = 413: 236 female, 177 male) from three different schools took tests on stoichiometry using the Testlet instrument, which was provided through a web developed previously (computerised). Rasch analysis model helped psychometric analysis regarding reliability, linear validity of students’ knowledge, and difficulty of items. This study reveals that Computerised Testlet able to identify students’ thinking processes. Besides, the psychometric analysis results showed that using Testlet to measure conceptual knowledge had good unidimensionality and reliability; it could be used to measure stoichiometric conceptual knowledge. Some questions needed to be revised as they proved unable to distinguish between high and low students’ conceptual knowledge; in general, the instrument could be used and provided a good function in analysing conceptual knowledge on the topic of stoichiometry.