Deep neural networks based user interface detection for mobile applications using symbol marker

Abstract

The UI storyboard is a design drawing for the development of mobile application. The designer uses a storyboard as a tool to help communication between the developer and the designer. However, the developer can make an unintended widget which is different from the intention of the designer. In this paper, we propose a DNN (Deep Convolutional Neural Network) based automatic UI detection method using symbol markers to improve the accuracy of UI identification. The symbol marker can be useful for UI detection and developing the UI Code.