Discovering buffer overflow vulnerabilities in the wild: an empirical study

Context: Reporters of security vulnerabilities possess rich information about the security engineering process. Goal: We performed an empirical study on reporters of buffer overflow vulnerabilities to understand the methods and tools used during the discovery. Method: We ran the study in the form of an email questionnaire with open ended questions. The participants were reporters featured in the SecurityFocus repository during two six-month periods; we collected 58 responses. Results: We found that in spite of many apparent choices, reporters follow similar approaches. Most reporters typically use fuzzing, but their fuzzing tools are created ad hoc; they use a few debugging tools to analyze the crash introduced by a fuzzer; and static analysis tools are rarely used. We also found a serious problem in the vulnerability reporting process. Most reporters, especially the experienced ones, favor full-disclosure and do not collaborate with the vendors of vulnerable software. They think that the public disclosure, sometimes supported by a detailed exploit, will put pressure on vendors to fix the vulnerabilities. But, in practice, the vulnerabilities not reported to vendors are less likely to be fixed. Conclusions: The results are valuable for beginners exploring how to detect and report buffer overflows and for tool vendors and researchers exploring how to automate and fix the process.