Cockpit Display Graphics Symbol Detection for Software Verification Using Deep Learning

Abstract

In Software Development Life-cycle, Verification and Validation plays a very important role, especially in the case of Safety-Critical Industries like Aerospace. Display dashboard consists of multiple static and dynamic objects having affine transformation, graphics overlap, shadows and less inter symbol discriminative features compared to natural images. Manual Software graphics verification is an error-prone and time-consuming activity. In this paper, we propose a novel software graphics verification pipeline to verify graphics symbols and alphanumeric objects as per Software requirements. To the best of our knowledge, our proposed approach is the first study on deep learning-based graphics symbol detection from complex synthetic background which requires high model accuracy. We experiment using Single-shot Multibox Detector (SSD) and You Only Look Once (YOLO v2) to detect different Graphical symbols from display simulator real-time captured video frames. These detected objects are further classified based on their nature. Objects containing alphanumeric digits can be recognized using Optical Character Recognition and dynamic symbols are detected using object detection to infer other properties. Finally, all the extracted properties can be compared with test expectations to verify their correctness. The result shows superior accuracy of the SSD algorithm over other state-of-the-art object detection algorithms for detecting real-time graphics symbols.