Prediction of the Testing Effort for the Safety Certification of Open-Source Software: A Case Study on a Real-Time Operating System

The reuse of Open Source Software (OSS) for safety-critical systems is seen with interest by industries, such as automotive, medical, and aerospace, as it enables shorter time-to-market and lower development costs. However, safety certification demands to supply evidence about OSS quality, and a gap analysis is needed to assess if the cost to produce certification evidence is worthwhile. This paper presents an empirical study on an open-source RTOS (RTEMS). The study investigates the relationship between software complexity and the effort to achieve a high test coverage, which is one of the most impacting activity for certification. The objective is to figure out if, and to what extent, it is possible to predict such effort preventively, by looking at software complexity metrics. This would enable a preliminary screening and benchmarking of OSS items, supporting strategic decision making. The study shows that combining metrics with classifiers can achieve a good prediction accuracy.