Likelihood function derivatives for a linear mixed model with compound symmetry assumption

Abstract

The paper explores the properties of linear mixed models with simple random effects of the form: yi = Xiβ + ZiYi + εi, i = 1, . . . ,M, Yi ∼ N(0, Ψ), εi ∼ Т(0, σ2I), where M is the number of distinct groups, each consisting of ni observations. Random effects Yi and within-group errors εi are independent across different groups and within the same group. β is a p-dimensional vector of fixed effects, Yi is a q-dimensional vector of random effects, and Xi and Zi are known design matrices of dimensions nixp and nixq, of fixed and random effects respectively. Vectors εi represent within-group errors with a spherically Gaussian distribution.Assuming a compound symmetry in the correlation structure of the matrix Ψ governing the dependence among within-group errors, analytical formulas for the first two partial derivatives of the profile restricted maximum likelihood function with respect to the correlation parameters of the model are derived. The analytical representation of derivatives facilitates the effective utilization of numerical algorithms like Newton-Raphson or Levenberg-Marquardt.The restricted maximum likelihood (REML) estimation is a statistical technique employed to estimate the parameters within a mixed-effects model, particularly in the realm of linear mixed models. It serves as an extension of the maximum likelihood estimation method, aiming to furnish unbiased and efficient parameter estimates, especially in scenarios involving correlated data. Within the framework of the REML approach, the likelihood function undergoes adjustments to remove the nuisance parameters linked to fixed effects. This modification contributes to enhancing the efficiency of parameter estimation, particularly in situations where the primary focus is on estimating variance components or when the model encompasses both fixed and random effects.