Cardinality Single Column Analysis for Data Profiling using an Open Source Platform

Abstract

Data quality is essential for an enterprise system. However, several problems can eradicate the quality of data. One of them is the unfiltered data received. To overcome this issue, data engineer usually handle this such data by deploying data profiling process. There are several tools available to do this process. Each tool has its advantages according to needs. The main focus of this research is to compare the analysis results of two open-source data profiling tools based on cardinality method. The tools are Pentaho Data Integration (PDI) and Data Cleaner. The results of this study indicate that Pentaho can search for median values and distinct values for the data performed by profiling, while data cleaners cannot search for these values. Thus that Pentaho Data Integration is more detailed and specific compared to Data Cleaner