Using deep learning to distinguish cryptic traits in animal colour patterns

Abstract

Colouration is often useful for distinguishing between closely related species, or between sexes of a given species. Nevertheless, sometimes colouration is so complex or non-descript to a human viewer that it cannot be used for delimitation. In this article, we propose the use of a Deep Learning (DL) algorithm to discriminate colour patterns of individuals belonging to different species, populations or sexes that are not discriminable by humans. Namely, we test the effectiveness of DL at distinguishing (i) between two species of the amphibian urodele Salamandrina, S. perspicillata and S. terdigitata, (ii) between two populations of S. perspicillata based on ventral colour pattern, and between (iii) two populations and (iv) sexes of the painted turtle Chrysemys picta based on plastron colouration. The classification algorithm performs well at distinguishing Salamandrina species (96.8% of the test set), and C. picta sexes (83.9%) and populations (76.7%), but has a lower performance on distinguishing populations of S. perspicillata (66.7%). Thus, DL is able to detect colour pattern differences between biological groups when humans cannot reliably distinguish them, representing a new way to examine animal colouration when it is highly variable or has only subtle differences among individuals, although DL does not output how patterns are similar or different. However, finding that colour differences occur can represent the first step of a longer analytical workflow based on possible explanatory hypotheses, or can allow researchers to save time in the opposite case.