Maximizing the accuracy of genetic variance estimation and using a novel generalized effective sample size to improve simulations.

Abstract

Key message: We developed an improved variance estimation that incorporates prediction error variance as a correction factor, alongside a novel generalized effective sample size to enhance simulations. This approach enables precise control of variance components, accommodating for more flexible and accurate simulations. Phenotypic variation in field trials results from genetic and environmental factors, and understanding this variation is critical for breeding program simulations. Additive genetic variance, a key component, is often estimated using linear mixed models (LMM), but can be biased due to improper scaling of the genomic relationship matrix. Here, we show that this bias can be minimized by incorporating prediction error variance (PEV) as a correction factor. Our results demonstrate that the PEV-based estimation of additive variance significantly improves accuracy, with root mean square errors orders of magnitude lower than traditional methods. This improved accuracy enables more realistic simulations, and we introduce a novel generalized effective sample size (ESS) to further refine simulations by accounting for sampling variation. Our method outperforms standard simulation approaches, allowing flexibility to include complex interactions such as genotype by environment effects. These findings provide a robust framework for variance estimation and simulation in genetic studies, with broad applicability to breeding programs.