Emmanuel Kabuga, Izzy Langley, Monica Arso Civil, John Measey, Bubacarr Bah, Ian Durbach
{"title":"相似性学习网络能独特识别四种海洋和陆地物种的个体","authors":"Emmanuel Kabuga, Izzy Langley, Monica Arso Civil, John Measey, Bubacarr Bah, Ian Durbach","doi":"10.1002/ecs2.70012","DOIUrl":null,"url":null,"abstract":"<p>Estimating the size of animal populations plays an important role in evidence-based conservation and management. Some methods for estimating population size rely on animals being individually identifiable. Traditionally, this has been done by marking physically captured animals, but increasingly, animals with distinctive natural markings are surveyed noninvasively using cameras. Animal reidentification from photographs is usually done manually, which is expensive, laborious, and requires considerable skill. An alternative is to develop computer vision methods that can support or replace the manual identification task. We developed an automated approach using deep learning to identify whether a pair of photographs is of the same individual or not. The core of the approach is a similarity learning network that uses paired convolutional neural networks with a triplet loss function to summarize image pairs and decide whether they are from the same individual. Prior to the main matching step, two additional convolutional neural networks perform image segmentation, cropping the animal object within the image, and orientation prediction, deciding which side of the animal was photographed. We applied the approach to four species, with images of the same individual often spanning several years: systematic surveys of bottlenose dolphins (<i>Tursiops truncatus</i>, 2008–2019) and harbor seals (<i>Phoca vitulina</i>, 2015–2019), a citizen science dataset of western leopard toads (<i>Sclerophrys pantherina</i>, unknown dates), and a publicly available repository of humpback whale images (<i>Megaptera novaeangliae</i>, unknown dates). For these species, our best-performing models were able to identify whether a pair of images were from the same individual or different individuals in 95.8%, 94.6%, 88.2%, and 83.8% of the cases, respectively. We found that triplet loss functions outperformed binary cross-entropy loss functions and that data augmentation and additional manual curation of training data provided small but consistent improvements in performance. These results demonstrate the potential of deep learning to replace or, more likely, support and facilitate manual individual identification efforts.</p>","PeriodicalId":48930,"journal":{"name":"Ecosphere","volume":null,"pages":null},"PeriodicalIF":2.7000,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ecs2.70012","citationCount":"0","resultStr":"{\"title\":\"Similarity learning networks uniquely identify individuals of four marine and terrestrial species\",\"authors\":\"Emmanuel Kabuga, Izzy Langley, Monica Arso Civil, John Measey, Bubacarr Bah, Ian Durbach\",\"doi\":\"10.1002/ecs2.70012\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Estimating the size of animal populations plays an important role in evidence-based conservation and management. Some methods for estimating population size rely on animals being individually identifiable. Traditionally, this has been done by marking physically captured animals, but increasingly, animals with distinctive natural markings are surveyed noninvasively using cameras. Animal reidentification from photographs is usually done manually, which is expensive, laborious, and requires considerable skill. An alternative is to develop computer vision methods that can support or replace the manual identification task. We developed an automated approach using deep learning to identify whether a pair of photographs is of the same individual or not. The core of the approach is a similarity learning network that uses paired convolutional neural networks with a triplet loss function to summarize image pairs and decide whether they are from the same individual. Prior to the main matching step, two additional convolutional neural networks perform image segmentation, cropping the animal object within the image, and orientation prediction, deciding which side of the animal was photographed. We applied the approach to four species, with images of the same individual often spanning several years: systematic surveys of bottlenose dolphins (<i>Tursiops truncatus</i>, 2008–2019) and harbor seals (<i>Phoca vitulina</i>, 2015–2019), a citizen science dataset of western leopard toads (<i>Sclerophrys pantherina</i>, unknown dates), and a publicly available repository of humpback whale images (<i>Megaptera novaeangliae</i>, unknown dates). For these species, our best-performing models were able to identify whether a pair of images were from the same individual or different individuals in 95.8%, 94.6%, 88.2%, and 83.8% of the cases, respectively. We found that triplet loss functions outperformed binary cross-entropy loss functions and that data augmentation and additional manual curation of training data provided small but consistent improvements in performance. These results demonstrate the potential of deep learning to replace or, more likely, support and facilitate manual individual identification efforts.</p>\",\"PeriodicalId\":48930,\"journal\":{\"name\":\"Ecosphere\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.7000,\"publicationDate\":\"2024-10-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ecs2.70012\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Ecosphere\",\"FirstCategoryId\":\"93\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/ecs2.70012\",\"RegionNum\":3,\"RegionCategory\":\"环境科学与生态学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ECOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ecosphere","FirstCategoryId":"93","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/ecs2.70012","RegionNum":3,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ECOLOGY","Score":null,"Total":0}
Similarity learning networks uniquely identify individuals of four marine and terrestrial species
Estimating the size of animal populations plays an important role in evidence-based conservation and management. Some methods for estimating population size rely on animals being individually identifiable. Traditionally, this has been done by marking physically captured animals, but increasingly, animals with distinctive natural markings are surveyed noninvasively using cameras. Animal reidentification from photographs is usually done manually, which is expensive, laborious, and requires considerable skill. An alternative is to develop computer vision methods that can support or replace the manual identification task. We developed an automated approach using deep learning to identify whether a pair of photographs is of the same individual or not. The core of the approach is a similarity learning network that uses paired convolutional neural networks with a triplet loss function to summarize image pairs and decide whether they are from the same individual. Prior to the main matching step, two additional convolutional neural networks perform image segmentation, cropping the animal object within the image, and orientation prediction, deciding which side of the animal was photographed. We applied the approach to four species, with images of the same individual often spanning several years: systematic surveys of bottlenose dolphins (Tursiops truncatus, 2008–2019) and harbor seals (Phoca vitulina, 2015–2019), a citizen science dataset of western leopard toads (Sclerophrys pantherina, unknown dates), and a publicly available repository of humpback whale images (Megaptera novaeangliae, unknown dates). For these species, our best-performing models were able to identify whether a pair of images were from the same individual or different individuals in 95.8%, 94.6%, 88.2%, and 83.8% of the cases, respectively. We found that triplet loss functions outperformed binary cross-entropy loss functions and that data augmentation and additional manual curation of training data provided small but consistent improvements in performance. These results demonstrate the potential of deep learning to replace or, more likely, support and facilitate manual individual identification efforts.
期刊介绍:
The scope of Ecosphere is as broad as the science of ecology itself. The journal welcomes submissions from all sub-disciplines of ecological science, as well as interdisciplinary studies relating to ecology. The journal''s goal is to provide a rapid-publication, online-only, open-access alternative to ESA''s other journals, while maintaining the rigorous standards of peer review for which ESA publications are renowned.