Inverse Kinematics and Temporal Convolutional Networks for Sequential Pose Analysis in VR

Drawing from a recent call to advance generalizability and causal inference in psychological science using contextually representative research designs [1], we introduce a conceptual framework that integrates techniques in machine perception of poses with VR-driven inverse kinematic character animation, leveraging the Unity game engine to mediate between the human user and the machine learner. This Computational Virtual Reality (C-VR) system contains the following components: a) Human motion capture (VR), b) Human to avatar character animation (inverse kinematics), c) character animation recordings (virtual cameras), d) avatar pose detection (OpenPose), d) avatar pose classification (SVM), and e) sequential avatar moving pose analyses (TCN). By leveraging the precision in representation afforded in virtual environments and agents and the precision in perception afforded in computer vision and machine learning in a unified system, we may take steps towards understanding a wider range of human complexity.