An attention-based loss function and synthetic minority oversampling technique for alleviating class imbalance in predicting diabetes

Abstract

Diabetes is a chronic disease due to higher blood sugar (or Glucose) levels in the blood. This study proposes a novel attention-based loss function and a lightweight artificial neural network (ANN) called Diabetic Lite (DB-Lite) for diabetes prediction in the Pima Indian Diabetes Dataset (PIDD). We show that the Pima dataset has many challenges. It is a small and imbalanced dataset; moreover, many features are non-linearly correlated in this dataset. The novelties of this research work are as follows: (i) A novel loss function of attention-based binary cross entropy (ABCE) is proposed for the first time to alleviate the statistical imbalance present within the Pima dataset. This ABCE loss function is incorporated in the DB-Lite model, which is trained from scratch. (ii) A Swish activation function is deployed in the hidden layer of DB-Lite instead of Rectified Linear Unit (ReLU) to deal with the non-linear dependency of features with the final outcome. (iii) The synthetic minority oversampling technique (SMOTE) is used as a pre-processing technique to mitigate the class imbalance problem from the Pima dataset. (iv) An adaptive learning rate is utilized while training the model to speed up the convergence of the DB-Lite model. Our final proposed framework has achieved 99.7% accuracy, 99.4% precision, 99.8% recall, and 99.6% F1 score in testing, which is the best result on this Pima dataset. The Welch t-testing (as a statistical hypothesis testing) and 10-fold cross-validation are utilized to prove the validity of the proposed loss function.