{"title":"A deep learning model for detecting and classifying multiple marine mammal species from passive acoustic data","authors":"Quentin Hamard , Minh-Tan Pham , Dorian Cazau , Karine Heerah","doi":"10.1016/j.ecoinf.2024.102906","DOIUrl":null,"url":null,"abstract":"<div><div>Underwater passive acoustics is used worldwide for multi-year monitoring of marine mammals. Yet, the large amount of audio recordings raises the need to automate the detection of acoustic events. For instance, the increasing number of Offshore Wind Farms (OWF) raises key environmental and societal issues relating to their impacts on wildlife. In this context, monitoring marine mammals along with information on their acoustic environment throughout the OWF life cycle is crucial. The objective of this study is to evaluate the ability of a single deep learning model to precisely detect and localize, in time and in frequency, the marine mammal sounds over a wide frequency range and classify them by species and sound types.</div><div>A broadband hydrophone, deployed at the Fécamp OWF (Normandy, France), recorded the underwater soundscape including sounds from marine mammals occurring in the area. To visualize these sounds, 15-s spectrograms were computed. From these images, dolphin (D) and porpoise (P) sounds were manually annotated, including different types of sounds: Click-Trains (D<sub>CT</sub>, P<sub>CT</sub>), Buzzes (D<sub>B</sub>, P<sub>B</sub>) and Whistles (D<sub>W</sub>). The spectrograms were then split into five-fold cross-validation datasets, each containing one half of manual annotations and one half of only background noise. A Faster R-CNN model was trained to precisely detect and classify the marine mammal sounds in the spectrograms.</div><div>Three model output configurations were used: (1) overall detection of marine mammals (presence vs. absence), (2) detection and classification of species (two classes: dolphin, porpoise) and (3) sound types (five classes: D<sub>CT</sub>, D<sub>B</sub>, D<sub>W</sub>, P<sub>CT</sub>, P<sub>B</sub>). For the simplest configuration (1) 15.4 % of the spectrogram dataset had detections while missing only 6.6 % of annotated spectrograms. For the more precise configurations, (2) and (3), the mean Average Precision (mAP) achieved were 92.3 % (2) and 84.3 % (3), and the macro average Area under the curve (AUC) 95.7 % (2) and 94.9 % (3).</div><div>This model will help to speed up the annotation processes, by reducing the spectrogram quantity to be manually analyzed and having time-frequency boxes already drawn. Several model parameters can be adjusted to trade off missed detections and false positives which need to be carefully considered and adapted to the problem. For instance, these adjustments would be particularly relevant depending on the human resources available to manually check the model detections and the criticality of missing marine mammal sounds. These models are promising, ranging from the simple detection of marine mammal presence to precise ecological inferences over the long term.</div></div>","PeriodicalId":51024,"journal":{"name":"Ecological Informatics","volume":"84 ","pages":"Article 102906"},"PeriodicalIF":5.8000,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ecological Informatics","FirstCategoryId":"93","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1574954124004485","RegionNum":2,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ECOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Underwater passive acoustics is used worldwide for multi-year monitoring of marine mammals. Yet, the large amount of audio recordings raises the need to automate the detection of acoustic events. For instance, the increasing number of Offshore Wind Farms (OWF) raises key environmental and societal issues relating to their impacts on wildlife. In this context, monitoring marine mammals along with information on their acoustic environment throughout the OWF life cycle is crucial. The objective of this study is to evaluate the ability of a single deep learning model to precisely detect and localize, in time and in frequency, the marine mammal sounds over a wide frequency range and classify them by species and sound types.
A broadband hydrophone, deployed at the Fécamp OWF (Normandy, France), recorded the underwater soundscape including sounds from marine mammals occurring in the area. To visualize these sounds, 15-s spectrograms were computed. From these images, dolphin (D) and porpoise (P) sounds were manually annotated, including different types of sounds: Click-Trains (DCT, PCT), Buzzes (DB, PB) and Whistles (DW). The spectrograms were then split into five-fold cross-validation datasets, each containing one half of manual annotations and one half of only background noise. A Faster R-CNN model was trained to precisely detect and classify the marine mammal sounds in the spectrograms.
Three model output configurations were used: (1) overall detection of marine mammals (presence vs. absence), (2) detection and classification of species (two classes: dolphin, porpoise) and (3) sound types (five classes: DCT, DB, DW, PCT, PB). For the simplest configuration (1) 15.4 % of the spectrogram dataset had detections while missing only 6.6 % of annotated spectrograms. For the more precise configurations, (2) and (3), the mean Average Precision (mAP) achieved were 92.3 % (2) and 84.3 % (3), and the macro average Area under the curve (AUC) 95.7 % (2) and 94.9 % (3).
This model will help to speed up the annotation processes, by reducing the spectrogram quantity to be manually analyzed and having time-frequency boxes already drawn. Several model parameters can be adjusted to trade off missed detections and false positives which need to be carefully considered and adapted to the problem. For instance, these adjustments would be particularly relevant depending on the human resources available to manually check the model detections and the criticality of missing marine mammal sounds. These models are promising, ranging from the simple detection of marine mammal presence to precise ecological inferences over the long term.
期刊介绍:
The journal Ecological Informatics is devoted to the publication of high quality, peer-reviewed articles on all aspects of computational ecology, data science and biogeography. The scope of the journal takes into account the data-intensive nature of ecology, the growing capacity of information technology to access, harness and leverage complex data as well as the critical need for informing sustainable management in view of global environmental and climate change.
The nature of the journal is interdisciplinary at the crossover between ecology and informatics. It focuses on novel concepts and techniques for image- and genome-based monitoring and interpretation, sensor- and multimedia-based data acquisition, internet-based data archiving and sharing, data assimilation, modelling and prediction of ecological data.