An Analysis Model of Buffer Overflow Vulnerability Based on FSM

Abstract

Buffer overflow vulnerabilities have been the most common form of software vulnerabilities. It is very difficult and time consuming to detect possible types of vulnerabilities from a program. This paper proposes an analysis model of buffer overflow vulnerability based on finite state machine (FSM). The model conducts static analysis on source code. And then it analyzes the formation of buffer overflow vulnerabilities and process of data overflow. For the two types of buffer overflow vulnerabilities caused by function call errors and loop copy errors, the corresponding vulnerability analysis model is designed. The vulnerability analysis model proposed in this paper is verified by two scenarios. The experimental results show that the model can detect buffer overflow vulnerability automatically and effectively.