Further thoughts on comparative analyses of coloration

Responding to our recent publication (Howell & Caro, 2024), Penteriani (2024) raises some important issues about independent measures used to tease out predictions about the social significance of coloration, the taxonomic level at which to conduct comparative analyses, and how to score colour patterns. For example, across terrestrial non-volant mammals, we found little evidence of social signalling involving coloration or colour patterns based on coarse social and ecological associations. Our independent measures were activity cycle, where we expected diurnal mammals to be more colourful; social group size, where we expected more social mammals to be more colourful; sexual dimorphism, with dimorphic species (showing evidence of sexual selection) expected to be more colourful; mating system, with polygynous species expected to be more colourful owing to the possible use of colour patches in male–male competition over mates; and congener overlap, where we expected species in danger of hybridization to be more colourful in order to signal species identity. Using these coarse measures, we found rather few significant associations, raising questions as to whether our independent measures were sufficiently sensitive to uncover signatures of social communication involving colour patterns. In part, our variables were dictated by the large breadth of species we examined. These were, in effect, ‘lowest common denominators’ that are documented for every species; they were not nuanced. Whether a solitary lifestyle means that individuals are less likely to signal socially is difficult to answer – we appreciate that these are not necessarily correlated.

The second issue is the taxonomic levels over which we analysed data. Findings at the Class level may not be reflected at the Order level; Order-level results may not be replicated at the Family level, and so on (Martins, 1996). Using comparative phylogenetic methods, many of our predictions did not hold up across the Class, or some Order levels for which we had sufficient data, even though we often based our hypotheses on associations already uncovered by others at Genus or Family levels. Thus for bears, conspicuous faces and chest bibs do seem to signal individual identity (Penteriani et al., 2020, 2023; although not tested phylogenetically owing to small sample size), but this association is washed out at the level of carnivores when other non-ursid species are included. What to do? Unfortunately, there is no clear solution. Higher taxonomic levels with greater numbers of species allow more statistical power and enable general conclusions to be formulated; lower taxonomic levels generate specific conclusions for certain groups, but species numbers can be so low for some groups that statistical tests are questionable. In our recent paper (Howell & Caro, 2024), we opted for the higher-level analyses; in others we have used Family-level analyses (Caro et al., 2014; Ortolani & Caro, 1996). Despite our higher-level attempt, we, like Penteriani (2024), value examining lower-level associations: mice are unlike bears, and duikers are unlike equids, both ecologically and morphologically.

A third point is that Howell and Caro's (2024) measures of pattern, which included adjacent blocks of colour, stripes and complex patterns, do not capture variations in shapes of colour patches in sun bears (Helarctos malayanus; Penteriani et al., 2020) and that may exist for giant panda (Ailuropoda melanoleuca) eye spots (unpublished data). Differences in these patterns may provide information on individual identity or age to conspecifics. Teasing out these proposals using phylogenetic analyses presents a challenge.

With some notable exceptions (e.g. West & Packer, 2002; Santana et al., 2012; Penteriani et al., 2020), most mammals do not rely on pelage coloration to communicate visually. One possible explanation is that many are solitary. If conspecifics are absent for much of an animal's life, it is reasonable to suggest that signalling might better involve odour, which lasts longer than a visual signal; the latter being a form of communication that is effective only when individuals can see each other. Nonetheless, there are other explanations for mammals being drab including the near ubiquity of dichromatic vision in mammals or intense predation pressure that many face.

Despite these quibbles, we think that Penteriani's (2024) critique is well placed, and we are pleased that he has drawn attention to these issues that dog phylogenetic comparisons.