Making AI accessible for forensic DNA profile analysis

Abstract

Deep learning has the potential to be a powerful tool for automating allele calling in forensic DNA analysis. Studies to date have relied on bespoke model architecture and painstaking manual annotations to train models, which makes it challenging for other researchers to work with these techniques. In this study, we explore the possibility of training a well-performing model using data gathered as part of casework, and employing a widely adopted architecture: the U-Net. In this approach, annotations are created from alleles called during casework. The model, dubbed ‘DNANet’, then classifies each scan point in the electropherogram (EPG) as part of an allele or non-allele, building on the task of segmentation in computer vision. We evaluate performance on unseen case data and on independent mixture research data, taking analyst annotations as ground-truth. We further compare DNANet’s performance with analyst performance on the research data, taking actual donor alleles as ground-truth. DNANet reached an F1 score of 0.971 on analyst annotated alleles on case data not seen during training, and 0.982 on the research data. On actual donor alleles, DNANet reached an F1 score of 0.962, equal to the F1 score computed from analyst annotations. Our results show that DNANet’s performance is comparable to human annotations following standard procedures. This illustrates the potential for obtaining good results with standard data and architecture. Future work may focus on what aspects of data, annotations or model architecture are key in shaping performance. We make our code, model weights and research data publicly available to aid the community. Lastly, we call for an effort to establish a standardized benchmark to aid in quantitative comparisons between allele calling systems.