Empowering robotic training with kinesthetic learning and digital twins in human–centric industrial systems

This paper presents a human-centric mixed reality (MR) collaborative training platform that employs a kinesthetic learning technique in industrial robotic training, specifically focusing on robot pick–and–place (RPP) operations. Collaborating with ABB Robotics Vietnam, we conducted a user study to investigate the user experiences and practical perceptions of university students and novice trainees via the human–centric training assessment. The study compares the traditional training (TT) RPP classroom as a conventional method with a new collaborative MR RPP training approach (N = 50). The MR training features a digital twin (DT) of ABB GoFa™ CRB–15000 collaborative robot in an immersive 360° Digital–Objects–Based Augmented Training Environment (360–ATE) using Microsoft HoloLens devices. The research evaluated the impact of MR and DT on human–robot interaction and collaboration, user experience, task performance, knowledge retention, and interpretation, as well as differences in perceptions between the two novice cohorts under each training condition. The primary research question explores “Whether the MR collaborative training platform with DT integration in 360–ATE can serve as an alternative approach for novice students and industrial trainees in RPP operations?”. The findings indicate that MR training is more engaging and effective in enhancing participant safety, confidence, and task performance, which also augments cognitive capabilities. The virtual contents on HoloLens, especially the DT, captured the attention and stimulated active learning abilities. Overall, participants in the MR cohort find the proposed training platform useful and easy to use. The platform has a positive influence on their intention to use similar 360–ATE–assisted training platforms in the future.