Deep learning to capture leaf shape in plant images: Validation by geometric morphometrics

Plant leaves play a pivotal role in automated species identification using deep learning (DL). However, achieving reproducible capture of leaf variation remains challenging due to the inherent “black box” problem of DL models. To evaluate the effectiveness of DL in capturing leaf shape, we used geometric morphometrics (GM), an emerging component of eXplainable Artificial Intelligence (XAI) toolkits. We photographed Ranunculus auricomus leaves directly in situ and after herbarization. From these corresponding leaf images, we automatically extracted DL features using a neural network and digitized leaf shapes using GM. The association between the extracted DL features and GM shapes was then evaluated using dimension reduction and covariation models. DL features facilitated the clustering of leaf images by source populations in both in situ and herbarized leaf image datasets, and certain DL features were significantly associated with biological leaf shape variation as inferred by GM. DL features also enabled leaf classification into morpho-phylogenomic groups within the intricate R. auricomus species complex. We demonstrated that simple in situ leaf imaging and DL reproducibly captured leaf shape variation at the population level, while combining this approach with GM provided key insights into the shape information extracted from images by computer vision, a necessary prerequisite for reliable automated plant phenotyping.