Tcherly:: A Teacher-facing Dashboard for Online Video Lectures

Abstract

The use of online video lectures in universities, primarily for content delivery and learning, is on the rise. Instructors ’ ability to recognize and understand student learning experiences with online video lectures, identify particularly difficult or disengaging content and thereby assess overall lecture quality can inform their instructional practice related to such lectures. This paper introduces Tcherly, a teacher-facing dashboard that presents class-level aggregated time series data on student s’ self-reported cognitive-affective states they experienced during a lecture. Instructors can use the dashboard to evaluate and improve their instructional practice related to video lectures. We report the detailed iterative prototyping design process of the Tcherly Dashboard involving two stakeholders (instructors and designers) that informed various design decisions of the dashboard, and also provide usability and usefulness data. We demonstrate, with real-life examples of Tcherly Dashboard use generated by the researchers based on data collected from six courses and 11 lectures, how the dashboard can assist instructors in understanding thei r students’ learning experiences and evaluating the associated instructional materials. decision-making. This paper demonstrates how stakeholders (instructors and designers) can be involved in the design process of such a dashboard to inform microlevel design decisions such as visualization, the format of dashboard elements, and supports required by instructors to make sense of the presented information (analytics). The evolution of the dashboard design through iterative prototyping with instructors and designers is demonstrated along with usability and usefulness evaluation results. • We present guidelines for instructors to use the dashboard based on data gathered from six courses and 11 lectures. The real-life examples of dashboard use demonstrate how to use dashboard features and visualizations in tandem to understand student learning experiences and evaluate the associated instructional materials.