A novel DLDRM: Deep learning-based flood disaster risk management framework by multimodal social media data.

Abstract

The impacted community and humanitarian organizations have used social media platforms extensively over the past 10 years to disseminate information during a disaster. Even though numerous researches have been conducted in recent times to categorize useful and non-informational posts on social media, the majority of these studies are unimodal, that is, they separately employed documented or pictorial information to improve deep learning (DL) approaches. In this research, a multimodal DL approach will be created by integrating the complementary data offered by the text and visual Twitter posts made by members of the affected community discussing the same occurrence. For the classification of multimodal disaster data, we suggested a novel DLDRM: DL-based disaster risk management structure. We contrast DLDRM with the most widely used bilinear multimodal models for visual question answering, including VGG 16, VGG 19, ResNet 50, DenseNet 121, and RegNet Y320. Accuracy, Precision, Recall, and F1-score were achieved utilizing DLDRM of 99%, 92.5%, 84.08%, and 98.5%. By emphasizing more pertinent aspects of text and image tweets, the proposed DL-based multimodal technique surpasses the present state-of-the-art fusion technique on the benchmark multimodal disaster dataset.