A new and almost perfectly accurate approximation of the eigenvalue effective population size of a dioecious population: comparisons with other estimates and detailed proofs

Abstract

The effective population size is an important concept in population genetics. It corresponds to a measure of the speed at which genetic drift affects a given population. Moreover, this is most of the time the only kind of population size that empirical population genetics can give access to. Dioecious populations are expected to display excesses of heterozygosity as compared to monoecious panmictic populations, as measured by Wright’s FIS. It can be shown that these excesses are negatively correlated with the population size. This is why FIS can be used to estimate the eigenvalue effective population size of dioecious populations. In this paper, we propose a new approximation that provides a very accurate estimate of the eigenvalue effective population size of a dioecious population as a function of the real population size. We then explore the accuracy of different FIS-based methods using the leading eigenvalue of transition matrices or coalescence. It appears that the eigenvalue-based method provides more accurate results in very small populations, probably due to approximations made by the coalescence approach that are less valid in such situations. We also discuss the applicability of this method in the field.