Do we betray errors beforehand? The use of eye tracking, automated face recognition and computer algorithms to analyse learning from errors

Abstract

Preventing humans from committing errors is a crucial aspect of man-machine interaction and systems of computer assistance. It is a basic implication that those systems need to recognise errors before they occur. This paper reports an exploratory study that utilises eye-tracking technology and automated face recognition in order to analyse test persons’ emotional reactions and cognitive load during a computer game and learning through trial and error. Computer algorithms based on machine learning and big data were tested that identify particular patterns of test persons’ gaze behaviour and facial expressions that antecede errors in a computer game. The results show that emotions and learning from errors are positively correlated and that gaze behaviour and facial expressions inform about the errors that follow. However, the algorithms still need to be improved through further studies to be suitable for daily use. This research is innovative in its use of mathematical formulae to operationalise learning through errors and the use of computer algorithms to predict errors in human behaviour in trial- and-error situations.