Combining Symbolic Reasoning and Deep Learning for Human Activity Recognition

Activity recognition (AR) plays an important role in situation aware systems. Recently, deep learning approaches have shown promising results in the field of AR. However, their predictions are overconfident even in cases when the action class is incorrectly recognized. Moreover, these approaches provide information about an action class but not about the user context, such as location and manipulation of objects. To address these problems, we propose a hybrid AR architecture that combines deep learning with symbolic models to provide more realistic estimation of the classes and additional contextual information. We test the approach on a cooking dataset, describing the preparation of carrots soup. The results show that the proposed approach performs comparable to state of the art deep models inferring additional contextual properties about the current activity. The proposed approach is a first attempt to bridge the gap between deep learning and symbolic modeling for AR.