Feature Relevancy Evaluation Based on Entropy Information

Abstract

with the huge data that is available from a simple raw dataset, the relevancy of features became important part in data mining. However, most comparison of classification result evaluates only the classification performance while compromising the quality of the attributes. Hence, this paper focused on the entropy evaluation based on bit per instance and compared using the two feature selection methods such as filter and wrapper, namely: InfoGain and wrapper Subset Evaluation using J48 algorithm. Evaluated features are both encoded to an expert machine classifier to perform the evaluation of the attributes that output rule. The overall accuracy in terms of classifiers performance gained by filter method is 97.9752% while Wrapper method gained 98.0422%. The result of Log-loss prior to the probabilities with respect to its entropy has both produced 0.2488 bits/instance of filter with 28 numeric attributes and wrapper method with 8 numeric attributes. On the other hand, the results of log-loss that shows the class complexity on scheme using Wrapper method has produced better result with 1.3381 bits/instance than Filter method with 6.2335 bits/instance. Lastly, Log loss information in this study produced sufficient information not only in the classifiers performance with respect to each classes but also to the model produced.