Visualization and Integration of Databases Using Self-Organizing Map

Abstract

With the growing computer networks, accessible data is becoming increasingly distributed.  Understanding and integrating remote and unfamiliar data sources are important data management issues.  In this paper, we propose to utilize self-organizing maps (SOM) clustering to aid with the visualization of similar columns, and integration of relational database tables and attributes based on the content. In order to accommodate heterogeneous data types found in relational databases,  we extended the TFIDF measure to handle, in addition to text, numerical attribute types for coincident meaning extraction.  We present a SOM clustering based visualization algorithm allowing the user to browse the heterogeneously typed database attributes and discover semantically  similar clusters. Additionally, we propose a new algorithm Common Item Based Classifier (CIBC) to smoothen the homogeneity of the clusters obtained by SOM. The discovered semantic clusters can significantly aid in manual or automated constructions of data integrity constraints in data cleaning or schema mappings in data integration.