A Model-Based Deep-Learning Approach to Reconstructing the Highly Articulated Flight Kinematics of Bats

Abstract

Bats are capable of highly dexterous flight maneuvers that rely heavily on highly articulated hand skeletons and malleable wing membranes. To understand the underlying mechanisms, large amounts of detailed data on bat flight kinematics are required. Conventional methods to obtain these data have been based on tracing landmarks and require substantial manual effort. To generate 3D reconstructions of the entire geometry of a flying bat in a fully automated fashion, the current work has developed an approach where the pose of a trainable articulated mesh template that is based on the bat's anatomy is optimized to fit a set of binary silhouettes representing views from different directions of the flying bat. This is followed by post-processing to smooth the reconstructed kinematics and simulate the non-rigid motion of the wing membranes. To evaluate the method, 10 flight sequences that represent several flight maneuvers (e.g., straight flight, takeoff, u-turn) and were recorded in a flight tunnel instrumented with 50 synchronized cameras have been reconstructed. A total of 4975 reconstructions are generated in this fashion and subject to qualitative and quantitative evaluations with promising results. The reconstructions are to be used for quantitative analyses of the maneuvering kinematics and the associated aerodynamics.