Intellitell: A Web-based Storytelling Platform for Emotion Recognition with Machine Learning

Abstract

Remote learning is relatively new for the education sector and is proving to be a challenge. Preschoolers are in a stage where they are highly active, and making sure that they are immersed in their educational activities is important. However, surveys and studies have shown that many students are not engaged in module-based distance learning methods. Higher engagement has been associated with higher emotional investment and higher chances of receiving positive learning outcomes. The goal of this paper was to develop a web-based storytelling platform, Intellitell, that gathered data on the student’s engagement. The paper assessed a preschooler’s engagement by gathering their emotional response using faceAPI.js during a storytelling activity using emotion match accuracy as a defining metric. The results showed that the emotion match accuracy of the students using the Intellitell webapp was 80.16% for defined data points. Further analysis showed that the ground truth is susceptible to distractions due to the low sample size, and classifying emotions that are similar like surprised and fearful is inconsistent without prior context to stimuli.