An Adaptive Method for Likelihood Optimization in Linear Mixed Models Under Constrained Search Spaces

Linear mixed effects models are highly flexible in handling correlated data by considering covariance matrices that explain variation patterns between and within clusters. For these covariance matrices, there exist a wide list of possible structures proposed by researchers in multiple scientific areas. Maximum likelihood is the most common estimation method in linear mixed models and it depends on the structured covariance matrices for random effects and errors. Classical methods used to optimize the likelihood function, such as Newton-Raphson or Fisher's scoring, require analytical procedures to obtain parametrical restrictions to guarantee positive definiteness for the structured matrices and it is not, in general, an easy task. To avoid dealing with complex restrictions, we propose an adaptive method that incorporates the so-called Hybrid Genetic Algorithms with a penalization technique based on minimum eigenvalues to guarantee positive definiteness in an evolutionary process which discards non-viable cases. The proposed method is evaluated through simulations and its performance is compared with that of Newton-Raphson algorithm implemented in SAS® PROC MIXED V9.4.