Sensitivity and feature importance of climate factors and evaluation of different machine learning models for predicting fire hotspots in Kalimantan, Indonesia

Abstract

Hotspots as indicators of forest fires capable of quickly monitoring large areas are often predicted using various machine learning methods. However, there is still little research that analyzes the sensitivity and feature importance of each predictor that forms a machine learning prediction model. This study evaluates and compares several machine learning methods to predict hotspots in Kalimantan. Using the most accurate machine learning model, each climate factor used as a predictor is analyzed for its sensitivity and feature importance. Some of the machine learning methods used include random forest, gradient boosting, Bayesian regression, and artificial neural networks. Meanwhile, several measures of sensitivity and feature importance used are variance-based, density-based, and distribution-based sensitivity indices, as well as permutation and Shapley feature importance. Evaluation of the ML model concluded that the Bayesian linear regression model outperformed other ML models, based on RMSE and explained variance score. Meanwhile, tree-based models, such as random forest and gradient boosting, are indicative of overfit. Based on the results of sensitivity analysis and feature importance, the number of dry days is the most important feature for the Bayesian linear regression model in predicting the number of hotspots in Kalimantan.