Research on Creating a Data Warehouse Based on E-Commerce

Abstract

With the popularization of the internet and the rapid development of science and technology, “online shopping” has become the norm in people's lives, and the e-commerce industry is booming, n addition, it has led to an increase in logistics. in today's business Wars, many companies strive for better development in enterprises of the same type, which continue to improve their information capabilities and level. This paper in order to solve the problems such as the increasing of massive data of e-commerce logistics and the phenomenon of data isolation in various business systems. The overall data warehouse is designed and constructed on the Hadoop cluster environment and data warehouse tool Hive is used to process data. Extraction of data from ETL, Sqoop and Flume tools is used for retrieving business data and log data and other aspects of ETL, we use Scala and Java to easily process and filter data and upload it to HDFS. The data warehouse is divided into levels and subject areas to simplify data management. Under the design of the entire system and data warehouse architecture, the conceptual, logical, and physical models of the data warehouse are developed and the star model is selected as a dimensional model. Finally, the application and implementation of data warehouse based on e-commerce logistics will be demonstrated. The development of a data warehouse based on e-commerce logistics not only ensures that e-commerce companies receive logistics information in a timely manner, but also forces decision makers to adjust logistics strategies in a timely manner based on data information, which can also improve user satisfaction and experience, and reduce costs.