A trigger-substrate model for smiling during an automated formative quiz: engagement is the substrate, not frustration

Introduction: Automated tutoring systems aim to respond to the learner's cognitive state in order to maintain engagement. The end-user's state might be inferred by interactive timings, bodily movements or facial expressions. Problematic computerized stimuli are known to cause smiling during periods of frustration. Methods: Forty-four seated, healthy participants (age range 18-35, 18 male) used a handheld trackball to answer a computer-presented, formative, 3-way multiple choice geography quiz, with 9 questions, lasting a total of 175 seconds. Frontal facial videos (10 Hz) were collected with a webcam and processed for facial expressions by CrowdEmotion using a pattern recognition algorithm. Interactivity was recorded by a keystroke logger (Inputlog 5.2). Subjective responses were collected immediately after each quiz using a panel of visual analogue scales (VAS). Results: Smiling was five-fold enriched during the instantaneous feedback segments of the quiz, and this was correlated with VAS ratings for engagement but not with happiness or frustration. Nevertheless, smiling rate was significantly higher after wrong answers compared to correct ones, and frustration was correlated with the number of questions answered incorrectly. Conclusion: The apparent disconnect between the increased smiling during incorrect answers but the lack of correlation between VAS frustration and smiles suggests a trigger-substrate model where engagement is the permissive substrate, while the noises made by the quiz after wrong answers may be the trigger.