Building Effective and Efficient Procedure for Preprocessing Marketplace Data

Abstract

Rapid development of digitalization have enforced National Statistics Offices to utilize big data as one of new sources for producing official statistics. An alternative source is marketplace data which now growing rapidly. Many challenges exist for transforming these massive datasets into statistics for public policy. This paper aims to explain the challenges of analyzing marketplace data and building effective and efficient preprocessing procedure to analyses big data which can be used for public policy. An optimal pipeline for preprocessing including validating, cleaning and aggregating marketplace data have been developed.