Human Sentiments and Associated Physical Actions Detection in Disasters with Deep Learning.

Abstract

In the study of emotions and activities, the increasingly growing development of social networks and the tendency of users to share their physical activities, opinions, expressions and perspectives in text, visual and audio content have opened up new possibilities and challenges. While the literature has widely examined sentiment and action interpretation of text streams, it is comparatively recent but challenging to evaluate sentiment and physical actions from visuals such as photos and videos together. This paper focuses human emotion with connected physical activity analysis in a socially critical area, namely social media disaster/catastrophe analysis. For disaster-related videos and photos in occluded areas, we propose multi-tagging sentiment and associated action analysis. We believe that the proposed approach will lead to more viable communities by benefiting multiple stakeholders, such as news broadcasters, emergency relief services, and the public in general.