BGICR: Bootstrap-guided iterative clustering refinement for enhanced high-dimensional psychological data analysis

Abstract

High-dimensional psychological data poses challenges due to noise, overlap, and projection distortion. This study presents Bootstrap-Guided Iterative Clustering Refinement (BGICR), a new framework developed to improve clustering in reduced-dimensional spaces. The proposed method uses silhouette-guided filtering and bootstrap sampling to iteratively remove ambiguous points through structural denoising, and it monitors validation scores until convergence. We used real-world psychological assessment data and applied four dimensionality reduction techniques: t-distributed stochastic neighbour embedding, uniform manifold approximation and projection, isometric mapping, and kernel principal component analysis. Results showed that BGICR consistently outperformed conventional clustering pipelines, with uniform manifold approximation and projection yielding the most distinct and well-separated clusters. Through adaptive iterations, the refinement improved the silhouette score to 0.7405, reduced the Davies–Bouldin index to 0.3914, increased the Calinski–Harabasz score to 3755.08, and achieved a Dunn index of 0.7689. Additional validation on synthetic data (Two-Moons) and biomedical datasets (LC25000 histopathological images) confirmed improved clustering quality with stable convergence and efficient runtime. Taken together, these results establish BGICR as a statistically grounded, noise-sensitive, and generalizable method for high-dimensional data analysis across psychological, synthetic, and biomedical domains.