Estimating genomic relationships of metafounders across and within breeds using maximum likelihood, pseudo-expectation–maximization maximum likelihood and increase of relationships

The theory of “metafounders” proposes a unified framework for relationships across base populations within breeds (e.g. unknown parent groups), and base populations across breeds (crosses) together with a sensible compatibility with genomic relationships. Considering metafounders might be advantageous in pedigree best linear unbiased prediction (BLUP) or single-step genomic BLUP. Existing methods to estimate relationships across metafounders $${\varvec{\Gamma}}$$ are not well adapted to highly unbalanced data, genotyped individuals far from base populations, or many unknown parent groups (within breed per year of birth). We derive likelihood methods to estimate $${\varvec{\Gamma}}$$ . For a single metafounder, summary statistics of pedigree and genomic relationships allow deriving a cubic equation with the real root being the maximum likelihood (ML) estimate of $${\varvec{\Gamma}}$$ . This equation is tested with Lacaune sheep data. For several metafounders, we split the first derivative of the complete likelihood in a term related to $${\varvec{\Gamma}}$$ , and a second term related to Mendelian sampling variances. Approximating the first derivative by its first term results in a pseudo-EM algorithm that iteratively updates the estimate of $${\varvec{\Gamma}}$$ by the corresponding block of the H-matrix. The method extends to complex situations with groups defined by year of birth, modelling the increase of $${\varvec{\Gamma}}$$ using estimates of the rate of increase of inbreeding ( $$\Delta F$$ ), resulting in an expanded $${\varvec{\Gamma}}$$ and in a pseudo-EM+ $$\Delta F$$ algorithm. We compare these methods with the generalized least squares (GLS) method using simulated data: complex crosses of two breeds in equal or unsymmetrical proportions; and in two breeds, with 10 groups per year of birth within breed. We simulate genotyping in all generations or in the last ones. For a single metafounder, the ML estimates of the Lacaune data corresponded to the maximum. For simulated data, when genotypes were spread across all generations, both GLS and pseudo-EM(+ $$\Delta F$$ ) methods were accurate. With genotypes only available in the most recent generations, the GLS method was biased, whereas the pseudo-EM(+ $$\Delta F$$ ) approach yielded more accurate and unbiased estimates. We derived ML, pseudo-EM and pseudo-EM+ $$\Delta F$$ methods to estimate $${\varvec{\Gamma}}$$ in many realistic settings. Estimates are accurate in real and simulated data and have a low computational cost.