A Multimodal Approach for Evaluating Algal Bloom Severity Using Deep Learning

Abstract

Harmful algal blooms (HABs) can have detrimental impacts on aquatic ecosystems, human health, and the economy. This paper presents a novel multimodal deep learning approach for assessing the severity levels of HABs, which will help to take necessary measures to mitigate the negative impacts. Unlike the other SOTA methods, the proposed method leverages three modalities: satellite image, elevation, and temperature data, to capture algal information. In particular, it utilizes an Attention-UNet-based encoder for satellite and elevation data, and a BiL-STM encoder for temperature data, to extract effective feature embeddings from respective modalities. In addition, we propose a geometric mean-based multimodal focal loss that modulates loss contributions of different modalities as a function of the confidence of different modalities. Our approach outperforms the SOTA unimodal and ensemble methods on tick-tick bloom (TTB) dataset, achieving a region-averaged root mean squared error (RA-RMSE) score of 0.8165.