PASTIS: A Collaborative Approach to Combine Heterogeneous Software Testing Techniques

Abstract

The fuzzing research field experienced outstanding advances over the past decade, making it a very effective approach for software testing. Dynamic Symbolic Execution (DSE) also called whitebox fuzzing is another approach that also evolved significantly and has the advantage of being able to solve very complex path conditions. Given fuzzing advances, knowing whether DSE is still relevant to help improving fuzzing coverage is an open research question. This paper aims at answering this question by empirically studying software testing techniques in a collaborative environment, and especially combining fuzzing and DSE. We show that in some cases DSE helps uncover some code locations not reached by fuzzers or can speed up the discovery of such coverage. We propose a combination of fuzzing and DSE into an ensemble fuzzing framework called PASTIS that helps in circumventing engines inner-working discrepancies. In this line, we propose a few DSE optimizations to maximize its ability to provide relevant inputs to fuzzers. One of them is a coverage strategy called prefixed-edge, which is an optimization over edge coverage. Finally, we show the advantages and roadblocks one can meet in trying to combine heterogeneous fuzzers and give insights on how to do it more efficiently.