To tune or not to tune? An approach for recommending important hyperparameters for classification and clustering algorithms

Machine learning algorithms are widely employed across various applications and fields. Novel technologies in automated machine learning ease the complexity of algorithm selection and hyperparameter optimization process. Tuning hyperparameters plays a crucial role in determining the performance of machine learning models. While many optimization techniques have achieved remarkable success in hyperparameter tuning, even surpassing human experts’ performance, relying solely on these black-box techniques can deprive practitioners of insights into the relative importance of different hyperparameters. In this paper, we investigate the importance of hyperparameter tuning by establishing a relationship between machine learning model performance and their corresponding hyperparameters. Our focus is primarily on classification and clustering tasks. We conduct experiments on benchmark datasets using six traditional classification and clustering algorithms, along with one deep learning model. Our findings empower users to make informed decisions regarding the necessity of engaging in time-consuming tuning processes. We highlight the most important hyperparameters and provide guidance on selecting an appropriate configuration space. The results of our experiments confirm that the hyperparameters identified as important are indeed crucial for performance. Overall, our study offers a quantitative basis for guiding automated hyperparameter optimization efforts and contributes to the development of better-automated machine learning frameworks.