Augmented Reality Text Translation: A Unity-Based Real-Time Approach

Abstract

Language barriers have remained as one of the biggest challenges when it comes to global communication. The need for text translation to happen in real time has been on the rise due to the idea of globalization in the past decades. However, there are very few well-developed applications on the market right now to serve the purpose of real-time text translation. This paper presents a Unity-based application that seamlessly translates text in real time in an augmented reality environment. It successfully combined Unity engine and C# scripts with services such as Google Cloud Vision and Google Translate to address the problem of cross-lingual understanding. The application utilizes a pre-trained machine learning model from Google Cloud Vision to recognize and translate texts and pins the translated text in relative positions on the camera as objects. The overall outcome of the application demonstrated the possibility of breaking down text language barriers. It also proved the potential usage of such technology in daily life. Users were able to view translated text objects through the perspective of augmented reality thus enabling a seamless translation experience in various environments.