Comparing linear and nonlinear reading assessments in PIRLS 2016

Abstract

Are linear paper-based and nonlinear computer-based reading comprehension measuring the same cognitive constructs? Nonlinear hypertext reading may demand different skills than linear text comprehension, as it requires synthesizing and integrating information from multiple sources. However, research on whether these two forms of comprehension represent distinct constructs remains limited. While conceptual distinctions exist between understanding a single linear text and processing multiple texts, empirical evidence supporting these differences is scarce. To address this gap, we examined the dimensional structure of paper-based linear and computer-based nonlinear text comprehension using data from the PIRLS 2016 international large-scale assessment. PIRLS 2016 evaluated reading comprehension at the end of grade 4, with each student completing both a linear paper-based test and a nonlinear electronic test. Comparing a unidimensional model, a 2-dimensional correlated factor model, and a bifactor IRT model across 16 countries and benchmarking entities showed that the bifactor model, which accommodates a general reading comprehension factor along with nonlinear reading as a specific factor, has a better fit than the other models. The ECVs of the general factor ranged from.80 to.89 across all the countries while the ECVs of the specific factor were between.07 and.18. Nevertheless, the specific factor reliabilities ranged from.36 to.58 which indicates that the variations in the specific factor cannot be just noise and could reflect variations in reading nonlinear texts. These findings revealed that although linear paper-based and nonlinear computer-based reading share a substantial amount of variance, as evidenced by a strong general reading comprehension factor, understanding nonlinear hypertexts requires specific skills that cannot be overlooked and should be explicitly addressed. The implications of the study for the testing and teaching of reading comprehension are discussed.