Web Log Analysis Based on Hadoop Technology

Abstract

The world is now in the information age. It is important to explore the valuable information from massive and diverse data. The analysis of the website log has great significance in practical application. Hadoop framework can provide reliable, scalable, distributed processing of large data sets. Hadoop series framework is utilized to conduct offline data processing and analysis for the website access log. The number of different browser, which visits to the master site access log, is counted. Different browser kernel, operating system, terminal type and other content access times are calculated and analyzed.