GADANN: A Virtual-to-Real knowledge transfer and adaptation method for edge devices

Abstract

Enabling Machine Learning capabilities on edge devices is crucial for supporting several automation scenarios. Given the low availability of real-world data for training, Virtual-to-Real knowledge transfer methods are often utilised for training Machine Learning models for deployment on edge devices located in the real world. However, the difference between the virtual and the real-world data hampers the post-deployment performance of the model. While Domain Adaptation-based methods allow a model to learn features shared across the virtual and the real worlds, the resulting model is static and too big to be used in conjunction with an edge device. TinyML allows the usage of Machine Learning models on resource-constrained devices by compressing the models into small transferable files. However, most of the existing TinyML onboard operations are restricted to drawing inferences and do not facilitate onboard training. To tackle these challenges, we propose $GADANN$, a TinyML-based Virtual-to-Real knowledge transfer method that facilitates onboard adaptation. $GADANN$ extends Machine Learning onboard the edge devices by leveraging Deep Neural Networks, Domain Adaptation, TinyML and Genetic Algorithms. The method has been tested successfully in a real-world setting by deploying it on a real robot in a warehouse prototype to identify pallets using computer vision. $GADANN$ can boost the accuracy of the robot by adapting to the environment in real-time.