Performance regression testing initiatives: a systematic mapping

Abstract

Context:

Issues related to the performance of software systems are crucial, as they have the potential to impede the effective utilization of products, compromise user satisfaction, escalate costs, and lead to failures. Performance regression testing has been identified as a prominent research domain, since it aims to prevent anomalies and substantial slowdowns.

Objective:

The objective of this paper is to examine recent approaches proposed in the literature concerning performance regression testing. Our interest lies in contributing insights that offer a forward-looking perspective on what is essential in this promising research domain.

Methods:

We carried out a systematic mapping study with the objective of gathering information on various initiatives related to performance regression testing. Our methodology follows the state-of-the-art guidelines for systematic mappings comprising planning, conducting, and reporting activities, thus obtaining a comprehensive set of selected studies.

Results:

Our selection includes 68 papers, and our analysis focuses on four key research questions, delving into (i) publication trends, (ii) developed approaches, (iii) conducted evaluations, and (iv) challenges. As a result of this investigation, we present a roadmap highlighting research opportunities.

Conclusion:

This flourishing research field entails a broad set of challenges, such as deciding the granularity of tests and the frequency of launching the performance regression process. Consequently, there is still much work to be undertaken to trade-off between the accuracy and the efficiency of capturing complex performance issues across diverse application domains and/or execution environments.