Human Activity Recognition Using Elliptical and Archimedean R-Vine Copulas with Multimodal Data

Abstract

We consider the problem of deep neural network (DNN)-based classification of the human activity using wearable sensors with multimodal data. The accelerometer and gyroscope sensors embedded in smart phones and smart watches are used to recognize the human activity. We employ a regular-vine (R-Vine) copula-based fusion of the high-level features extracted from the neural networks. In particular, we consider bivariate copulas from two different families, namely elliptical and Archimedean, to build the underlying R-Vine tree structure. Comparison of the two families are brought out using the publicly available STISEN data set. To evaluate the efficacy of these algorithms in a real-time scenario, we created a data set called PESHAR using MPU-6050 module with a Raspberry-pi. For the given data set, we found that the elliptical family of R-Vine structure has proved to be better in terms of ${F_{1}}$ scores and confusion matrices, as compared to the Archimedean family.