Language of actions: A generative model for activity recognition and next move prediction from motion sensors

Abstract

Increasing use of motion sensors in wearable and mobile devices has fuelled the development of new computational models to detect and monitor the context of the people via streaming data from those devices. A particular task is the recognition of current activity from motion signals acquired from miniature inertial sensors, such as accelerometers, embedded in wearable devices. The problem is formally defined as classification of a single-source triaxial motion signal into one of pre-defined categories of human activities. In this study, we propose a data processing framework based on a generative action model, which is inspired by language models in text processing, to understand actions and explain resulting activity. We show that the model can be used for several tasks such as activity recognition from a completed action or next move prediction in an incomplete action with known activity. The framework was tested on three different benchmark datasets, where the signals were collected from accelerometers worn in chest or wrist and labelled according to different activities relevant to position of the sensor placed. The model achieved 97.7% accuracy for recognizing hand activities using wrist-worn sensors, and 97.8–99% accuracy for recognizing whole-body activities using chest-worn sensors. The experimental results indicate the proposed model can provide an easily interpretable means of activity recognition and outperform many of the existing solutions in terms of classification accuracy. Furthermore, the model provides a strong baseline for next move prediction in an action, which may find applications in robotic simulations, human–computer interaction and synthetic data generation.