A Novel Feature Selection Approach and Feature Weight Adjustment Technique in Text Classification

Abstract

Feature selection and feature weight calculating are key preprocesses in text classification. A new feature selection approach based on average interaction gain(AIG) is presented and a new feature weight adjustment technique(WA) taking inter-class distribution and intra-class distribution into consideration is presented too. Then a new approach combining AIG with WA called AIG-WA is presented. In the following experiments, we use a support vector machine(SVM) classifier to compare the performance of AIG and AIG-WA with the commonly used feature selection algorithms. Better performances are obtained when applying this method on Chinese text dataset provided b Fudan Database Center.