Using Cognitive Models to Develop Digital Twin Synthetic Known User Persona

A recurring challenge in user testing is the need to obtain a record of user interactions that is large enough to reflect the different biases of a single user persona while accounting for temporal and financial constraints. One way to address this need is to use digital twins of user personas to represent the range of decisions that could be made by a persona. This paper presents a potential use of cognitive models of user personas from a single complete record of a persona to test the web-based decision support system, ALFRED the BUTLER. ALFRED the BUTLER is a digital cognitive assistant developed to generate recommended articles for users to review and evaluate relative to a priority information request (PIR).Interaction data for three different user personas for the ALFRED the BUTLER system were created: the Early Terminator, the Disuser, and the Feature Abuser. These three personas were named after the type of interaction they would have with the data and were designed to represent different types of human-automation user interactions as outlined by Parasuraman & Riley (1997). The research team operationalized the definitions of use, misuse, disuse, and abuse to fit the current context. Specifically, the Early Terminator represented misuse by no longer meaningfully interacting with the system once a search criterion was met whereas the Disuser represented disuse by never using a certain feature. The Feature Abuser represented abuse by excessively using a single feature when they should be using other features. Each member of the research team was assigned a user persona, given a briefing related to their persona, and instructed to rate 250 articles as either relevant (thumbs up), irrelevant (thumbs down), or neutral (ignore). Subsequently, a cognitive model of the task was built. Cognitive models rely on mechanisms that capture human cognitive processes such as memory, learning, and biases to make predictions about decisions that humans would be likely to make (Gonzalez & Lebiere, 2005). To construct the cognitive model, we relied on the Instance-Based Learning (IBL) Theory (Gonzalez et al., 2003), a cognitive theory of experience-based decision making. The data for each user’s previous actions were added to the model’s memory to make predictions about the next action the user would be likely to make (thumbs up, thumbs down, or ignore an article). The model was run 100 times for each persona, with the 250 articles presented in the same order as they were judged by the persona. The results indicate an overall model prediction accuracy of the persona’s decisions above 60%. Future work will focus on refining and improving the model's predictive accuracy The authors discuss future applications, one of which is using this type of cognitive modeling to help create synthetic datasets of persona behaviors for evaluation and training of machine learning algorithms.ReferencesGonzalez, C., & Lebiere, C. (2005). Instance-based cognitive models of decision-making.Gonzalez, C., Lerch, J. F., & Lebiere, C. (2003). Instance‐based learning in dynamic decision making. Cognitive Science, 27(4), 591-635.Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human factors, 39(2), 230-253.