Data Preparation for User Profiling from Traffic Log

Abstract

This paper presents our current work on traffic log processing. Our goal is to find an approach to modeling user behaviour based on their behavioural patterns. Since the amount of input data we have is really large, effective preprocessing is crucial for the profiling to provide significant results. This paper presents our approach to restricting the input data with respect to its relevance. We use histogram clustering to identify sets of users with similar frequencies of communication; entropy and TF-IDF (term frequency - inverse document frequency) help to select destinations that are relevant for a given set of users. The main profiling is done with preprocessed data and our experiments show that this approach to restricting the input has a positive impact on the significance of results.