An Approach to GUI Test Scenario Generation Using Machine Learning

Abstract

With the fast advance of artificial intelligence technology and data-driven machine learning techniques, more and more AI approaches are applied in software engineering activities, such as coding, testing and etc.. Conventionally, test engineers use manual testing tools to test mobile apps and deliver products. Object detection technology like YOLO is widely used in image processing these days. Inspired from this, on the basis of detecting GUI elements using machine learning models, we propose an automated approach to GUI test scenario generation based on mockup diagrams. The list of possible scenarios can be visualized using NetworkX which can indicate the feasibility and effectiveness of the proposed approach.