The Patterson-Price-Reich's rule of population structure analysis from genetic marker data

Delineating population structure from the marker genotypes of a sample of individuals is now routinely conducted in the fields of molecular ecology, evolution and conservation biology. Various Bayesian and likelihood methods as well as more general statistical methods (e.g. PCA) have been proposed to detect population structure, to assign sampled individuals to discrete clusters (subpopulations), and to estimate the admixture proportions of each sampled individual. Regardless of the methods, the power of a structure analysis depends on the strength of population structure (measured by F_ST) relative to the amount of marker information (measured by NL, where N and L are the numbers of sampled individuals and loci respectively). Patterson, Price and Reich (2006) proposed that population structure is unidentifiable when data size D = NL is smaller than $1/F_{ST}^2$ and quickly becomes identifiable easily with an increasing D or F_ST when $D>1/F_{ST}^2$. In this study, I investigated this phase change PPR rule by analysing both simulated genomic data and empirical data by four likelihood admixture analysis methods. The results show that the PPR rule is largely valid, but the accuracy of a structure analysis is also affected by the number of subpopulations K. A more complicated population structure with a larger K requires a larger $NL{F}_{ST}^2$ to resolve accurately. For a given $NL{F}_{ST}^2$ above the PPR threshold value of 1, increasing L and decreasing N is advantageous over increasing N and decreasing L in improving admixture estimation accuracy.