A forward k-means algorithm for regression clustering

We propose a novel forward k-means algorithm for regression clustering, where the “forward” strategy progressively partitions samples from a single cluster into multiple ones, using the current optimal clustering solutions as initialization for subsequent iterations, thereby ensuring a deterministic result without any initialization requirements. We employ the mean squared error from the fitted clustering results as a criterion to guide partition optimization, which not only ensures rapid convergence of the algorithm to a stable solution but also yields desirable theoretical results. Meanwhile, we also suggest a difference-based threshold ridge ratio criterion to consistently determine the number of clusters. Comprehensive numerical studies are further conducted to demonstrate the algorithm's efficacy.