Convolutional neural network hyperparameter optimization applied to land cover classification

Abstract

In recent times, machine learning algorithms have shown great performance in solving problems in different fields of study, including the analysis of remote sensing images, computer vision, natural language processing, medical issues, etc. A well-prepared input dataset can have a huge impact on the result metrics. However, a correctly selected hyperparameter combined with neural network architecture could highly increase the final metrics. Therefore, the hyperparameters optimization problem becomes a key issue in a deep learning algorithm. The process of finding a suitable hyperparameter combination could be performed manually or automatically. Manual search is based on previous research and requires enormous human efforts. However, there are many automated hyperparameter optimization methods have been successfully applied in practice. The automated hyperparameter tuning techniques are divided into two groups: black-box optimization techniques (such as Grid Search, Random Search) and multi-fidelity optimization techniques (HyperBand, BOHB). The most recent and promising among all approaches is BOHB which, which combines both Bayesian optimization and bandit-based methods, outperforms classical approaches, and can run asynchronously with given GPU resources and time budget that plays a vital role in the hyperparameter optimization process. The previous study proposed a convolutional deep learning neural network for solving land cover classification problems in the EuroSAT dataset. It was found that adding spectral indexes NDVI, NDWI, and GNDVI with RGB channels increased the result accuracy (from 64.72% to 84.19%) and F1 (from 63.89 % to 84.05%) score. However, the convolutional neural network architecture and hyperparameter combination were selected manually. The research optimizes convolutional neural network architecture and finds suitable hyperparameter combinations applied to land cover classification problems using multispectral images. The obtained results must increase result performance compared with the previous study and given budget constraints.