Reinforcement learning with digital human models of varying visual characteristics

Digital Human Modelling (DHM) is rapidly emerging as one of the most cost-effective tools for generating computer-based virtual human-in-the-loop simulations. These help better understand individual and crowd behaviour under complex situations. For tasks such as target search and wayfinding, the eye is the primary channel for processing perceptual information and decision making. Existing experimental human studies in the literature have highlighted the relationship between the field of vision, visual acuity, accommodation, and its effect on visual search performance. This paper presents a methodology for the simulation of visual behaviour in target search and a wayfinding task by employing DHM as a reinforcement learning agent with functional vision characteristics. We used Unity 3D game engine to build the DHM and virtual workspace, Unity ML-Agents package to realise its connection with TensorFlow, and the Proximal Policy Optimization (PPO) algorithm to train DHM in finding a target through intensive reinforcement learning (RL). For the functional vision system, we have considered three human-inspired vision personas: (i) ‘good vision’, (ii) ‘poor vision’ type 1 (low acuity like), and (iii) ‘poor vision’ type 2 (high myopia like). We have compared the emergent behaviour of DHM for each of the three personas and RL training performance. The results conclude that simulating reinforcement learning agents with varying vision characteristics can evaluate their impact on visual task performance.