Deep Uncertainty Interpretation in Dyadic Human Activity Prediction

Abstract

We propose a deep learning framework to analyse the uncertainty associated with dyadic human activities at a small temporal granularity. Such time-slice analysis is able to infer human behaviours from short-term observations. Instead of classifying time-slices into k classes of activities, we report to what degree of certainty each activity is occurring from definitely not occurring to definitely occurring. To this end, we extract CNN-based unary probabilities and pairwise relations between body joints. The unary term gives cues on the local appearance while the pairwise term captures the contextual relations between the parts. We extract the features from each frame in a timeslice and examine different temporal aggregation schemes to generate a descriptor for the whole time-slice. Evaluations on the TAP dataset which is well-suited for time-slice activity analysis demonstrate the effectiveness of our approach for the task of uncertainty analysis in activity prediction.