Data mining and analysis in depth. case study of Qafqaz University HTTP server log analysis

Abstract

The Internet Services, Web and Mobile Applications, Pervasive Communication widely available today meeting many of our needs and stimulating production of tremendous amounts of data. Over 90% of this information is unstructured, what means data does not have predefined structure and model. Generally, unstructured data is useless unless applying data mining or data extraction techniques. At the same time, just in case if we are able to process and understand data, this data worth anything, otherwise it becomes useless. Although, small part of this huge amount is structured (logs) or semi-structured (email, website), it is difficult to process and manage this data without advanced data analytics techniques. This paper provides an example of applying Data Mining and Analysis techniques on the data generated by HTTP Server Logs. Experimental results show that proposed analysis approach based on Regular Expressions is highly efficient and flexible. Results of such analysis are highly beneficial for any company which concerns about efficiency of their Internet-presence giving them important information based on the real data.