Applying Deep Learning to Public Health: Using Unbalanced Demographic Data to Predict Thyroid Disorder

Abstract

This study investigates the use of Deep Neural Learning to predict propensity for disease from demographic information alone, with thyroid disease as the test application. The imbalanced dataset of 747,301 samples contained 13 demographic predictor variables that were not known to be associated with the disease, and had much missing information. A TensorFlow feed-forward neural network was trained to predict thyroid disease. Different activation functions and a variety of up-sampling and down-sampling methods were employed. The lift statistic was used to evaluate success in identifying patients with a propensity for thyroid disease. The DNN model outperformed the Random Forest model with a 36.63% improvement in the lift statistic. These results suggest that deep learning may be successfully employed to select candidates for early intervention for improved health outcomes, utilizing a large dataset with only minimal demographic variables, similar to datasets that are held by the marketing arms of healthcare providers.