Enhancing big data analysis in IoT applications and optimizing the performance of machine learning models using hybrid dimensionality optimization approach

Abstract

The proliferation of Internet of Things (IoT) applications generates high-dimensional datasets characterized by substantial velocity, variety, and complexity, imposing severe computational constraints on machine learning systems. Such data's high dimensionality complicates identifying meaningful correlations among features. Thus, high-dimensional datasets pose substantial challenges for machine learning, as the abundance of variables tends to obscure meaningful correlations and hinder practical data analysis, particularly regarding computational resource consumption (e.g., memory usage), processing time, and machine learning models' training efficiency and performance. Dimensionality reduction techniques address these challenges by decreasing the number of input variables and preserving the intrinsic structure of the data while alleviating computational burdens. Nevertheless, most contemporary methods are optimized for either linear or nonlinear data patterns, but rarely both. Hybrid strategies integrating linear and nonlinear reduction techniques have increasingly addressed these constraints. Specifically, the combination of Principal Component Analysis (PCA) as a preprocessing stage with Restricted Boltzmann Machines (RBMs) offers a complementary solution, wherein PCA condenses the feature space into a lower-dimensional representation, thereby improving training efficiency and enabling RBMs to capture complex nonlinear dependencies with enhanced convergence and generalization. While this combination can theoretically exploit the data's linear and nonlinear characteristics, conventional PCA-RBM frameworks often struggle to retain essential local manifold structures, limiting their effectiveness in capturing the full complexity of real-world datasets. This study addresses these challenges by proposing a novel hybrid dimensionality reduction framework that integrates PCA's global linear projection capabilities with RBMs' nonlinear feature learning strengths through an adaptive graph regularization mechanism that preserves critical local manifold properties, which address the limitations of conventional PCA-RBM combinations. The adaptive regularization mechanism ensures that proximate data points in input space retain similarity in the reduced feature space, effectively bridging global and local structure preservation. Compared to conventional methods, experimental validation demonstrates superior performance across multiple evaluation metrics, including data reduction efficiency, classification accuracy, precision, recall, and F-score. The framework addresses critical limitations in high-dimensional data processing while maintaining model performance, establishing a methodologically significant contribution to dimensionality reduction techniques applicable across scientific disciplines handling complex IoT-generated datasets. Our findings indicate that dimensionality reduction constitutes a viable and efficacious approach to simplifying datasets without significantly compromising performance.