Development of A Virtual Environment to Realize Human-Machine Interaction of Forklift Operation

This study presents an experimental concept to develop realistic Human-Machine Interaction (HMI) for a Virtual Environment (VE) and a novel evaluation methodology of such system. Such evaluation is motivated by the need to facilitate transfer of model/knowledge from VE to the Real Environment (RE), where it is crucial for the VE to trigger similar user behavior as in the RE. This paper discusses the application of such concept to evaluate interactions of forklift operation in the VE. First, a Virtual Reality (VR) forklift simulator is developed using motion capture and 3D reconstruction methods to mimic HMI of the real forklift operation. Then, the Dynamic Time Warping (DTW) algorithm is used for temporal evaluation of operation behaviors in VE and RE. Results of DTW (i.e. distance and correlation) are used as objective measures to evaluate fidelity of VE during forklift operations on the simulator. Results suggest the proposed forklift simulator triggers operation behavior which is similar (highly correlated) to that of real forklift operation. The contributions of this paper are (a) the novel VR forklift simulator system to realize interactions of real forklift in the VE, and (b) the proposed objective measures for temporal evaluation of the fidelity of VE.