Work in progress: Audio reflections provide evidence of metacognition during students' problem solving attempts

Creating instructional materials that facilitate students' metacognition (activities such as planning, monitoring, and evaluating) and critical thinking is challenging for instructors of first year engineering students. Students come into these programs at various levels of academic preparation, and students may experience cognitive overload when asked to simultaneously use mathematical skills and higher-order thinking skills. In order to demonstrate the effectiveness of new approaches to introducing complex, relevant contexts and skills, methods must be developed that can provide evidence of metacognition and critical thinking skills. These methods can also facilitate research into problem-solving strategies. The overall goal of this project is to design problems for first year students that introduce complex real world topics that are effective for building problem-solving skills for students at all levels of academic preparation. The study presented here is the preliminary qualitative analysis of students' audio commentary in which they reflect on their written problem solutions in a first year engineering course. These reflections were performed within 24 hours of completing written work. Our analysis provides evidence of metacognition retrospectively, as well as during the post-hoc think-aloud commentary. This research methodology effectively identifies student problem-solving strategies, including cognitive, metacognitive and procedural information, and provides evidence that think-aloud protocols are useful in eliciting metacognition and critical thinking.