A Fitness Monitoring System based on Fusion of Visual and Sensorial Information

We present a method that recognizes exercising activities performed by a single human in the context of a real home environment. Towards this end, we combine sensorial information stemming from a smartphone accelerometer, with visual information from a simple web camera. Low-level features inspired from the audio analysis domain are used to represent the accelerometer data, while simple frame-wise features are used in the visual channel. Extensive experiments prove that the fusion approach achieves 95% of overall performance when user calibration is adopted, which is a 4% performance boosting compared to the best individual modality which is the accelerometer data.