Performance Testing in Software Development: Getting the Developers on Board

Society is increasingly dependent on software-intensive systems that are required to interact with a huge number of users and respond in timely manner. Failing that often results in users and consumers not being able to access the advertised services or products, or worse, citizens unable to access vital services provided by the state. In such systems, failures are more likely to be caused by performance issues than by faulty implementation of some features [3, 1]. Modern development processes for general-purpose software systems typically focus on managing complexity to deliver correctly functioning software on time, and best software development practices frown upon premature optimization. With other aspects of software design and construction put above performance concerns, performance becomes a secondary concern that only needs to be addressed if the system’s performance turns out to be unsatisfactory. This contrasts with real-time systems, where meeting realtime performance requirements is essential, and performance is a primary design concern that permeates the development process and the resulting system as a whole. Consequently, the overall performance of the system is a concern that cannot be addressed locally—it must be designed into the system, and strictly controlled throughout its construction. Simply adopting the process of real-time system development for the development of general-purpose systems is not possible. The size, complexity, and the depth of the software stack used to build general-purpose software-intensive systems typically dwarfs that of the special-purpose missionor safety-critical real-time systems. The level of control that can be exerted over individual elements of real-time systems either does not scale, or is not possible at all, in addition to performance requirements being usually much less precise, and not easily expressed in terms of latencies or deadlines. The productivity of developers during development is also an important aspect. Where real-time systems limit development flexibility to maintain control over performance (by avoiding many features of modern runtime platforms unless their performance impact can be sufficiently controlled),