Jonas Håkansson, Brooke L. Quinn, Abigail L. Shultz, Sharon M. Swartz, Aaron J. Corcoran
{"title":"将基于深度学习的新型 3D 摄像工作流程应用于蝙蝠飞行","authors":"Jonas Håkansson, Brooke L. Quinn, Abigail L. Shultz, Sharon M. Swartz, Aaron J. Corcoran","doi":"10.1111/nyas.15143","DOIUrl":null,"url":null,"abstract":"<p>Studying the detailed biomechanics of flying animals requires accurate three-dimensional coordinates for key anatomical landmarks. Traditionally, this relies on manually digitizing animal videos, a labor-intensive task that scales poorly with increasing framerates and numbers of cameras. Here, we present a workflow that combines deep learning–powered automatic digitization with filtering and correction of mislabeled points using quality metrics from deep learning and 3D reconstruction. We tested our workflow using a particularly challenging scenario: bat flight. First, we documented four bats flying steadily in a 2 m<sup>3</sup> wind tunnel test section. Wing kinematic parameters resulting from manually digitizing bats with markers applied to anatomical landmarks were not significantly different from those resulting from applying our workflow to the same bats without markers for five out of six parameters. Second, we compared coordinates from manual digitization against those yielded via our workflow for bats flying freely in a 344 m<sup>3</sup> enclosure. Average distance between coordinates from our workflow and those from manual digitization was less than a millimeter larger than the average human-to-human coordinate distance. The improved efficiency of our workflow has the potential to increase the scalability of studies on animal flight biomechanics.</p>","PeriodicalId":8250,"journal":{"name":"Annals of the New York Academy of Sciences","volume":null,"pages":null},"PeriodicalIF":4.1000,"publicationDate":"2024-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/nyas.15143","citationCount":"0","resultStr":"{\"title\":\"Application of a novel deep learning–based 3D videography workflow to bat flight\",\"authors\":\"Jonas Håkansson, Brooke L. Quinn, Abigail L. Shultz, Sharon M. Swartz, Aaron J. Corcoran\",\"doi\":\"10.1111/nyas.15143\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Studying the detailed biomechanics of flying animals requires accurate three-dimensional coordinates for key anatomical landmarks. Traditionally, this relies on manually digitizing animal videos, a labor-intensive task that scales poorly with increasing framerates and numbers of cameras. Here, we present a workflow that combines deep learning–powered automatic digitization with filtering and correction of mislabeled points using quality metrics from deep learning and 3D reconstruction. We tested our workflow using a particularly challenging scenario: bat flight. First, we documented four bats flying steadily in a 2 m<sup>3</sup> wind tunnel test section. Wing kinematic parameters resulting from manually digitizing bats with markers applied to anatomical landmarks were not significantly different from those resulting from applying our workflow to the same bats without markers for five out of six parameters. Second, we compared coordinates from manual digitization against those yielded via our workflow for bats flying freely in a 344 m<sup>3</sup> enclosure. Average distance between coordinates from our workflow and those from manual digitization was less than a millimeter larger than the average human-to-human coordinate distance. The improved efficiency of our workflow has the potential to increase the scalability of studies on animal flight biomechanics.</p>\",\"PeriodicalId\":8250,\"journal\":{\"name\":\"Annals of the New York Academy of Sciences\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":4.1000,\"publicationDate\":\"2024-04-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1111/nyas.15143\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Annals of the New York Academy of Sciences\",\"FirstCategoryId\":\"103\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1111/nyas.15143\",\"RegionNum\":3,\"RegionCategory\":\"综合性期刊\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MULTIDISCIPLINARY SCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of the New York Academy of Sciences","FirstCategoryId":"103","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/nyas.15143","RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MULTIDISCIPLINARY SCIENCES","Score":null,"Total":0}
Application of a novel deep learning–based 3D videography workflow to bat flight
Studying the detailed biomechanics of flying animals requires accurate three-dimensional coordinates for key anatomical landmarks. Traditionally, this relies on manually digitizing animal videos, a labor-intensive task that scales poorly with increasing framerates and numbers of cameras. Here, we present a workflow that combines deep learning–powered automatic digitization with filtering and correction of mislabeled points using quality metrics from deep learning and 3D reconstruction. We tested our workflow using a particularly challenging scenario: bat flight. First, we documented four bats flying steadily in a 2 m3 wind tunnel test section. Wing kinematic parameters resulting from manually digitizing bats with markers applied to anatomical landmarks were not significantly different from those resulting from applying our workflow to the same bats without markers for five out of six parameters. Second, we compared coordinates from manual digitization against those yielded via our workflow for bats flying freely in a 344 m3 enclosure. Average distance between coordinates from our workflow and those from manual digitization was less than a millimeter larger than the average human-to-human coordinate distance. The improved efficiency of our workflow has the potential to increase the scalability of studies on animal flight biomechanics.
期刊介绍:
Published on behalf of the New York Academy of Sciences, Annals of the New York Academy of Sciences provides multidisciplinary perspectives on research of current scientific interest with far-reaching implications for the wider scientific community and society at large. Each special issue assembles the best thinking of key contributors to a field of investigation at a time when emerging developments offer the promise of new insight. Individually themed, Annals special issues stimulate new ways to think about science by providing a neutral forum for discourse—within and across many institutions and fields.