Study of Various Dimensionality Reduction and Classification Algorithms on High Dimensional Dataset

Abstract

A potential drawback of huge data is that it makes analysis of the data hard and also computationally infeasible. Health care, finance, retail, and education are a few of the data mining applications that involve very high-dimensional data. A large number of dimensions introduce a popular problem of “Curse of Dimensionality”. This problem makes it difficult to perform classification and engenders lower accuracy of machine learning classifiers. This paper computes a threshold value (35%) to which if the data is reduced, the best accuracy can be obtained. Further, this research work considers an image dataset of very high dimensions on which different dimensionality reduction techniques such as PCA, LDA, and SVD are performed to find out the best dimension fit for an image dataset. Also, various ML classification algorithms, such as Logistic Regression, Random Forest Classifier, Naive Bayes, and SVM are applied to find out the best classifier for the dimensionally reduced dataset. Finally, this research work has concluded that, PCA+SVM, LDA+Random Forest, and SVD+SVM have produced the best results out of all the possible combinations from the comparative study.