A post‐processing framework for assessing BirdNET identification accuracy and community composition

IF 1.8 3区 生物学 Q1 ORNITHOLOGY
Ibis Pub Date : 2024-09-13 DOI:10.1111/ibi.13357
Michael C. Thompson, Mark J. Ducey, John S. Gunn, Rebecca J. Rowe
{"title":"A post‐processing framework for assessing BirdNET identification accuracy and community composition","authors":"Michael C. Thompson, Mark J. Ducey, John S. Gunn, Rebecca J. Rowe","doi":"10.1111/ibi.13357","DOIUrl":null,"url":null,"abstract":"Passively collected acoustic data have become increasingly common in wildlife research and have prompted the development of machine‐learning approaches to extract and classify large sets of audio files. BirdNET is an open‐source automatic prediction model that is popular because of its lack of training requirements for end users. Several studies have sought to test the accuracy of BirdNET and illustrate its potential in occupancy modelling of single or multiple species. However, these techniques either require extensive statistical knowledge or computational power to be applied to large datasets. In addition, there is a lack of comparisons of occupancy and community composition calculated using BirdNET and typical field methods. Here we develop a framework for assessing the accuracy of BirdNET using generalized linear mixed models to determine species‐specific confidence score thresholds. We then compare community composition under our model and another post‐processing approach to field data collected from co‐located point count surveys in northeastern Vermont. Our framework outperformed the other post‐processing method and resulted in species composition similar to that of point count surveys. Our work highlights the potential mismatch between accuracy and confidence score and the importance of developing species‐specific thresholds. The framework can facilitate research on large acoustic datasets and can be applied to output from BirdNET or other automatic prediction models.","PeriodicalId":13254,"journal":{"name":"Ibis","volume":null,"pages":null},"PeriodicalIF":1.8000,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ibis","FirstCategoryId":"99","ListUrlMain":"https://doi.org/10.1111/ibi.13357","RegionNum":3,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ORNITHOLOGY","Score":null,"Total":0}
引用次数: 0

Abstract

Passively collected acoustic data have become increasingly common in wildlife research and have prompted the development of machine‐learning approaches to extract and classify large sets of audio files. BirdNET is an open‐source automatic prediction model that is popular because of its lack of training requirements for end users. Several studies have sought to test the accuracy of BirdNET and illustrate its potential in occupancy modelling of single or multiple species. However, these techniques either require extensive statistical knowledge or computational power to be applied to large datasets. In addition, there is a lack of comparisons of occupancy and community composition calculated using BirdNET and typical field methods. Here we develop a framework for assessing the accuracy of BirdNET using generalized linear mixed models to determine species‐specific confidence score thresholds. We then compare community composition under our model and another post‐processing approach to field data collected from co‐located point count surveys in northeastern Vermont. Our framework outperformed the other post‐processing method and resulted in species composition similar to that of point count surveys. Our work highlights the potential mismatch between accuracy and confidence score and the importance of developing species‐specific thresholds. The framework can facilitate research on large acoustic datasets and can be applied to output from BirdNET or other automatic prediction models.
用于评估鸟网识别准确性和群落构成的后处理框架
被动采集的声学数据在野生动物研究中越来越常见,这也促使人们开发机器学习方法来提取大量音频文件并对其进行分类。BirdNET 是一个开源的自动预测模型,因其对终端用户没有培训要求而广受欢迎。有几项研究试图测试 BirdNET 的准确性,并说明其在单个或多个物种占位建模方面的潜力。然而,这些技术要么需要丰富的统计知识,要么需要强大的计算能力才能应用于大型数据集。此外,使用 BirdNET 和典型野外方法计算的鸟类栖息地和群落组成也缺乏比较。在此,我们建立了一个评估 BirdNET 准确性的框架,使用广义线性混合模型来确定特定物种的置信分阈值。然后,我们将我们的模型和另一种后处理方法下的群落组成与佛蒙特州东北部同地点点计数调查收集的实地数据进行比较。我们的框架优于另一种后处理方法,得出的物种组成与点计数调查相似。我们的工作凸显了准确度和置信度之间潜在的不匹配,以及制定特定物种阈值的重要性。该框架可促进对大型声学数据集的研究,并可应用于 BirdNET 或其他自动预测模型的输出。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Ibis
Ibis 生物-鸟类学
CiteScore
4.60
自引率
9.50%
发文量
118
审稿时长
6-12 weeks
期刊介绍: IBIS publishes original papers, reviews, short communications and forum articles reflecting the forefront of international research activity in ornithological science, with special emphasis on the behaviour, ecology, evolution and conservation of birds. IBIS aims to publish as rapidly as is consistent with the requirements of peer-review and normal publishing constraints.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信