Collaborative edge learning-assisted interactive avatar generation framework for IoT-driven metaverse

Abstract

The Beyond 5G (B5G) wireless networks offer massive Internet of Things (IoT) integration with metaverse services, leveraging high data rates and low latency capabilities. Avatars are essential components in the metaverse platform. Creating an interactive avatar that utilizes IoT-based human pose estimation by utilizing centralized deep learning (DL) poses significant challenges. Federated learning (FL) offers a solution by enabling local training on edge devices and collaboratively producing a reliable model. This study proposes edge learning-assisted human activity recognition (HAR) using the FL technique and integrates it with an IoT-driven intelligent metaverse platform to create an interactive avatar. The HAR framework captures human gestures using infrared array sensor devices and recognizes activities with a lightweight hybrid model called AvatarNet. An enhanced data distribution and reputation-aware (iDDR) client selection scenario is implemented to identify potential clients and improve model performance. Furthermore, a connection module based on JavaScript and the WebSocket protocol has been developed to integrate the HAR framework with the Unreal Engine (UE) metaverse platform. The proposed model was tested using our infrared-based HAR and public datasets, outperforming state-of-the-art in accuracy and model complexity. The measurements show that the proposed model achieves an accuracy of 96.49%, precision of 94.84%, recall of 94.78%, F1 score of 94.80%, and MFLOPs calculation of 0.0431.