UAV-based water pollutants detection and classification framework using multi-modal and multi-sensor ensemble learning

The massive increment in water pollutants due to the release of plastic, industrial, and household waste has threatened the delicate balance of ecosystems and the well-being of human life. Therefore, detection and monitoring of such water pollutants have become an essential task for the widespread and open surface water bodies. Recent advancements in UAVs with Computer Vision (CV) models and communication technologies have given the scope to automate the process of pollutant monitoring in such surface water bodies, minimizing human intervention. This paper presents a comprehensive framework integrating UAVs for autonomous data collection and pollutant classification. The customized YOLOv5 model is utilized for both the classification and detection of water pollutants, enhancing efficiency and accuracy. Moreover, we propose a multi-modal feature extraction module that uses Vision Transformer (ViT), YOLOv5, and NodeMCU sensors to create a comprehensive data representation. The extracted features are then classified using an ensemble model combining TabNet and XGBoost, improving the overall classification performance. An image dataset for water pollutant detection has been prepared using video sequences captured by a UAV-based camera at different zoom levels and altitudes. The results show that the proposed model performed better than the MobileNet, YOLOv4, YOLOv5s, and YOLOv8 in terms of both the response time and the mAP of (\(94.7\%\)) for Algae, (\(96.7\%\)) for trash, and (\(94.3\%\)) for the classification of pollutants of multi-classes. This work aims to advance the deployment of UAVs for environmental monitoring, providing an efficient and scalable solution for water pollutant detection.