Portable and Self-Powered Sensing AI-Enabled Mask for Emotional Recognition in Virtual Reality.

Abstract

With the increasing development of metaverse and human-computer interaction (HMI) technologies, artificial intelligence (AI) applications in virtual reality (VR) environments are receiving significant attention. This study presents a self-sensing facial recognition mask (FRM) utilizing triboelectric nanogenerators (TENG) and machine learning algorithms to enhance user immersion and interaction. Various TENG negative electrode materials are evaluated to improve sensor performance, and the efficacy of a single sensor is confirmed. For accurate facial movement and emotion detection, different machine learning algorithms are assessed, leading to the selection of an advanced data processing method with a two-layer long short-term memory model, which achieves 99.87% accuracy. The practical applications of the FRM system in virtual reality, including psychotherapy and HMI scenarios, are validated through mathematical models. Additionally, a digital twin-based monitoring platform is developed using 5G, database, and visualization technologies to oversee the user status. Overall, these innovative approaches overcome the limitations of existing face recognition technologies, including environmental interference and high cost, compared with other facial recognition technologies.