Fusing Visual Features and Metadata to Detect Flooding in Flickr Images

Abstract

Social media platforms such as Flickr have become a source of information for the assessment of natural disasters, for instance assisting in flood mapping. Visual features and textual metadata have been used to identify natural disasters in social media images, however, they have often been used separately. Here, we fuse these two modes together using two fusion methods and deep learning to identify flood images in the MediaEval 2017 dataset. A novel backpropagation technique, Direct Backpropagation (DBP) is used to train a neural network for the classification. The results show that the fusion methods improve the classification accuracy compared to their individual counterparts. We compare our proposed learning method with other baseline methods and find it producing highest classification results. For external evaluation, the results are compared with MediaEval 2017 methods, where our methods outperform most of them.