Predicting cardiovascular risk factors from facial & full body photography using deep learning

Abstract Introduction The early and easy detection of pathological cardiovascular phenotypes can lead to an early medical intervention and thus slow or limit the development of cardiovascular diseases. As full body photographs are easily obtainable without the need of medical expertise, this modality holds the potential to be viable for screening of populations. Purpose Utilizing data from a population-based study, we examined the possibility to detect cardiovascular risk factors from total body photographs using deep learning. Methods A population-based cohort study was utilized. The first data release provides facial and full body photographs in dermatologic standard poses of 6500 participants (median age 62.0 years, 49.6% men) and corresponding cardiovascular risk factors. Here, we focus on the most prevalent ones: smoking status (prevalence: 19.0%), hypertension (35.3%) and diabetes (8.2%). Here we employ a 2D-Convolutional Resnet-18 Neural Network for predicting the risk factors. It receives the full body image, the facial image and age and sex as input. We compare this to a logistic regression model only including sex and age. Logistic Regression and Neural Network are employed in a 5-fold validation scheme and t-tested for significance. Results Our model provided a good detection of arterial hypertension (AUC 0.711, CI 0.684–0.739), while a logistic regression on age and sex yielded a significantly worse prediction (AUC 0.681, CI 0.679– 0.683, p<0.05). Additionally, it made a good detection of a positive smoking status (AUC 0.733, CI 0.711–0.754), being significantly better than a respective logistic regression on age and sex (AUC 0.598, CI 0.597–0.6, p<0.001). Lastly, it classified diabetes well (AUC 0.744, CI 0.724–0.764, p<0.001) in comparison to the logistic regression (AUC 0.681, CI 0.679–0.683, p<0.001). Conclusion The presence of cardiovascular risk factors can be detected from a total body photograph. As total body photographs can be easily obtained with a majority of digital cameras, including smart phones, this model represents a potentially widely applicable diagnostic tool to easily screen large parts of the population for relevant cardiovascular risk factors, making an early medical intervention possible. Funding Acknowledgement Type of funding sources: None. Figure 1