On the Use of a Semantic Segmentation Micro-Service in AR Devices for UI Placement

Abstract

Augmented Reality (AR) technology is now increasingly applied in various fields, which can bring an unprecedented immersive experience and rich interaction to the application field. However, complex interactions and informative interfaces impose a long learning curve and burden on users. Making the AR experience more intelligent to reduce redundant operations is one solution to enhance the user experience. One potential research direction is seamlessly combining the two fields of machine learning and AR. This paper proposes using semantic segmentation to assist automatic information placement in AR using a case study within precision agriculture as an example. The precise location of the crop area in the user view is determined by semantic segmentation, which helps to place information in the AR environment automatically. The dataset used for machine learning model construction consists of 242 farmland images. Four semantic segmentation techniques are proposed and bench-marked against each other. The results show that the Attention U-Net deep neural network has the highest recognition accuracy, reaching 91.9%. An AR automatic information placement prototype using Attention U-Net has been developed to run on tablets utilising a micro-service approach. This work demonstrates how AR user interfaces could be placed correctly within the real world, which traditionally has been an understudied area of research within AR and is essential for future AR games and Enterprise applications. As such, this solution has potential usage in all areas of AR application.