Measuring In-Task Emotional Responses to Address Issues in Post-Task Questionnaires

When evaluating interactive information retrieval (IIR) interfaces, it is common to collect data using subjective measures such as satisfaction, ease of use, usefulness, and user engagement. However, as these are collected post-task, they serve as surrogate measures for what occurred in the midst of the search activities. Further, such approaches may be subject to recency effects, where the last action in the search process influences the searchers’ opinions about the overall process. With recent improvements in facial emotion classification approaches, we propose that measuring emotional responses may provide a better indication of what is happening throughout search tasks. In this research, we present an approach for collecting real-time emotional responses during a search activity using consumer-grade front-facing cameras and a method of aligning these with search interface feature use. To validate the effectiveness of the approach, we have conducted a controlled laboratory study in which we manipulated the quality of the search results in order to determine if we can detect expected emotional responses, whether search behaviours influencing these emotional responses, and whether recency effects are present in post-task measures. The preliminary results of this study show that our approach is reliable for detecting emotional responses when searchers experience positive and negative emotions throughout the search process, isolate which interactive elements were used when positive and negative emotional responses were experienced, and illustrate how recency effects are present in post-task measures. Our upcoming study will investigate how our approach can be used to evaluate novel search interfaces. We will develop a novel search interface and evaluate it using our approach. Finally, we will create a dashboard to monitor academic literature. Using the same approach, we will demonstrate our approach can extend beyond traditional search interfaces and into more general interface assessment.