Estimating Body Fat from Depth Images: Hand-Crafted Features vs Convolutional Neural Networks

In this paper, we compare different approaches to estimate body fat percentages from simple depth images that can be captured by low-cost sensors. We implemented two frameworks, one based on hand-crafted features, using simple image processing methods to estimate directly from images a set of body measurements (e.g. areas, lengths girths), and one based on Convolutional Neural Networks, applying a direct regression from the grayscale maps representing the body depth, based on a pretrained networks. With these frameworks, we evaluated the fat percentage predictions obtained with the different methods on depth images of 350 subjects with known body composition estimated with a DXA scanner. Depth images were generated by extracting the z-buffer from the renderings of the 3D body scan models acquired on the group of subjects. In our validation experiments, we evaluated the effect of different simulated acquisition setups, parameters settings, different image preprocessing and data-augmentation procedures and the addition of priors on height and weight on the prediction accuracy. Furthermore, since the dataset used is composed of professional sportsmen and a control group, we evaluated also the ability of both frameworks of predicting the sport practiced by the subjects with a cross-validation experiment. In specific, we propose a customized ResNet50 regressor to evaluate the whole body fat percentage of the subjects directly from the depth acquisitions. Using the same input data, we also set up a neural classifier to predict the sport category of the athlets. Despite the limited numbers of subjects and the restricted variability of body types (all males, Caucasian, with a small number of obese), the results obtained are promising and can be considered a first step towards the development of quick and cheap body fat estimation tools that can be extremely useful for sport, health and fitness applications.