Coveringcerts: Combinatorial Methods for X.509 Certificate Testing

Abstract

Correct behaviour of X.509 certificate validation code in SSL/TLS implementations is crucial to ensure secure communication channels. Recently, there have been major efforts in testing these implementations, namely frankencerts and mucerts, which provide new ways to generate test certificates which are likely to reveal errors in the implementations of X.509 validation logic. However, it remains a significant challenge to generate effective test certificates. In this paper, we explore the applicability of a prominent combinatorial method, namely combinatorial testing, for testing of X.509 certificates. We demonstrate that combinatorial testing provides the theoretical guarantees for revealing errors in the certificate validation logic of SSL/TLS implementations. Our findings indicate that the introduced combinatorial testing constructs, coveringcerts, compare favorably to existing testing methods by encapsulating the semantics of the validation logic in the input model and employing combinatorial strategies that significantly reduce the number of tests needed. Besides the foundations of our approach, we also report on experiments that indicate its practical use.