An Efficient Web Log Filter Scheme for Web Abnormal Behaviors

With the rapid development of technology, network services are becoming more complex and changeful. To protect the security and privacy of these network services, check and analyze abnormal behaviors actively becomes very important. In order to meet the need of check and analyze abnormal behaviors actively for routine security check, we try to find the data of known attacks and anomaly behaviors, propose a web log filter scheme for web abnormal behaviors which aims to quickly anomaly detection and also provides accuracy. The scheme uses the signature rules of PHPIDS to match, preprocesses network logs to find suspicious logs and form a feature matrix, reduces the dimensionality of matrix using random projection, uses Mahalanobis distance to identify outliers and calculate an anomaly score of the outliers. If the log line is too different, we flag it anomaly, until all of the logs are checked. In order to get the better outcome, we use the data of real-world company to test the scheme and find the suitable parameter. In addition, the advantage of the scheme is simple to implement easily, fast and without losing too much accuracy and does not need to clean training data.