Using virtual prototypes and metamorphic testing to verify the hardware/software-stack of embedded graphics libraries

Abstract

Embedded graphics libraries are part of the Firmware (FW) of embedded systems and provide complex functionalities optimized for specific hardware. After unit testing of embedded graphics libraries, integration testing is a significant challenge, in particular since the hardware is needed to obtain the output image as well as the inherent difficulty in defining the reference result. In this paper, we present a novel approach focusing on integration testing of embedded graphic libraries. We leverage Virtual Prototypes (VPs) and integrate them with Metamorphic Testing (MT). Metamorphic Testing (MT) is a software testing technique that uncovers faults or issues in a system by exploring how its outputs change under predefined input transformations, without relying on explicit oracles or predetermined results. In combination with virtualizing the displays in VPs, we even eliminate the need for physical hardware. This allows us to develop a Metamorphic Testing (MT) framework automating the verification process. In our evaluation, we demonstrate the effectiveness of our Metamorphic Testing (MT) framework. On an extended RISC-V Virtual Prototype (VP) for the GD32VF103VBT6 platform, we found 15 distinct bugs for the widely used TFT_eSPI embedded graphics library, confirming the strength our approach. We finish the evaluation of our Metamorphic Testing (MT) approach by discussing the achieved structural coverage for function, line and branch coverage.