Large-Scale Social Multimedia Analysis

Abstract

The Internet is abundant with opinions, sentiments, and reflections of the society about products, brands, and institutions hidden under tons of irrelevant and unstructured data. This work addresses the contextual augmentation of events in social media streams in order to fully leverage the knowledge present in social multimedia by making three major contributions. First, a global study of the Twitter Firehose is presented. To our knowledge this is the first study of this kind and comprehension providing valuable insights about variability of tweets with respect to multimedia content. The results for more than one billion tweets show the great potential of the stream for many application domains. As a second key contribution, a fully automated system was developed for the augmentation of social multimedia with contextual information on a large scale. The system trawls multimedia content from Twitter and performs a multi-modal analysis on it. The analysis considers temporal, visual, textual, geographical, and user-specific dimensions. Third, we present a near-duplicate detection approach based on deep learn- ing to detect the most frequent images being propagated through Twitter during events