A New Feature Hashing Approach Based on Term Weight for Dimensional Reduction

Abstract

Machine learning models usually face a problem when encountered with large scale text dataset. Such kind of dataset produces sparse features of a high-dimensional, which makes it complex or infeasible to process by the learning models. Feature hashing is a dimensional reduction technique commonly used in the pre-processing phase to overcome the aforementioned problem. However, models performance are negatively affected due to the inherited so-called collisions that occur during the hashing process. In this study, we proposed a new Feature hashing approach that hashes similar features to the same bin based on their weight known as "weight term" while minimizing certain collisions. The approach effectively reduces the collisions between dissimilar features, thus improving model performance. The experiment results conducted on binary and multi-class classification datasets with a very high number of sparse features show that the proposed approach achieved competitive performance compared with the conventional FH.