Uncovering Student Strategies for Solving Scientific Inquiry Tasks: Insights from Student Process Data in PISA

Abstract

The advancement of technology has led to a growing interest in assessing scientific inquiry within digital platforms. This shift towards dynamic and interactive inquiry assessments enables researchers to investigate not only the accuracy of student responses (product data) but also their steps and actions leading to those responses (process data). This is done by analyzing computer-generated log files that capture student activity during the assessment. The present study leverages this opportunity by drawing insights from student log files of the Programme for International Student Assessment (PISA). It demonstrates the potential of process data in uncovering typically unobserved students’ problem-solving processes by focusing on two critical scientific inquiry skills: coordinating the effects of multiple variables and coordinating a theory with evidence. This study presents two examples for analyzing process data. The first example examined data from the PISA field trial study and showcased the advantage of using a process mining approach to visualize the sequence of students’ steps and actions in conducting investigations. The second example linked student log files and questionnaire data from the PISA 2015. It applied latent profile analysis to identify unique patterns of students’ inquiry performance and examined their relationships to their school-based inquiry experiences. Findings from both examples indicate that students often encounter considerable challenges in solving complex inquiry tasks, especially in applying multivariable reasoning and constructing scientific explanations. This study highlights the profound potential of process data in facilitating a deeper understanding of how students interact with scientific inquiry tasks in a digital-based environment.