Comparison of cohort-based identical-by-descent (IBD) segment finding methods for endogamous populations

Segments of DNA that are inherited from a common ancestor are referred to as identical-by-descent (IBD). Because these segments are inherited, they not only allow us to study population characteristics and the sharing of rare variants but also understand the hidden familial relationships within populations. Over the past two decades, various IBD finding algorithms have been developed using hidden Markov model (HMM), hashing and extension, and Burrows-Wheeler Transform (BWT) approaches. In this study, we investigate the utility of pedigree information in enhancing the efficacy of IBD finding methods for endogamous populations. With the increasing prevalence of computationally efficient sequencing technology and proper documentation of pedigree structures, we expect complete pedigree information to become readily available for more populations. While IBD segments have been used to reconstruct pedigrees [1], because we now have access to the pedigree, it is a natural question to ask if including pedigree information would substantially improve IBD segment finding for the purpose of studying inheritance. Our contributions center around the proposition of two types of IBD finding algorithms for reducing the number of false positives in the detected IBD segments. Both methods analyze the familial relationships between cohorts of individuals who are initially hypothesized to share IBD segments. Our first algorithm is inspired by a k-nearest neighbors (KNN) algorithm [2] where we perform outlier detection on the cohort of IBD-sharing individuals. The metric for proximity is determined by the kinship coefficient evaluated from the pairwise relationships between individuals from the cohort. Our second algorithm is inspired by the Bonsai algorithm [3] and uses multiple hypothesis tests to evaluate if an individual has much more IBD than is expected by chance. Bonsai IBD detection algorithm first divides the pedigree into multiple cohorts of family members with no shared individuals, proceeds to pick the two cohorts with the most shared IBD, and performs a hypothesis test between individuals in the first cohort against everyone in the second cohort. If the hypothesis test is rejected, we remove the individual from the cohort, recompute the common ancestor, and recurse on the remaining individual and the new cohort. Essentially, we account for recombination rates on top of Bonsai's hypothesis tests computations. Our algorithms are evaluated against simulations of an endogamous Amish population to determine their efficacy in removing false positive IBD segments.