Quantifying heteroscedasticity in linear models using quantile locally weighted scatterplot smoothing intervals.

Abstract

Ordinary least squares (OLS) estimation, which is frequently applied in psychology, assumes constant variance of errors across predictor levels. This assumption is known as homoscedasticity, whereas its violation is referred to as heteroscedasticity. In categorical predictors, heteroscedasticity can be quantified by calculating the ratio of variances across groups. For continuous predictors, diagnostic residual plots are often used to assess whether the assumption had been met, but there is currently no measure that can quantify the amount of heteroscedasticity in an interpretable way. In this study, we have developed and evaluated a measure that constructs a quantile locally weighted scatterplot smoothing interval (QLI) around the residuals and estimates the linear, quadratic, cubic, and quartic change in the width of this interval as a function of the predictor or the fitted values. Furthermore, we evaluated simple linear models under different patterns of heteroscedasticity in a simulation to provide benchmark values of QLI estimates associated with inadequate control over false-positive results, loss of power, and loss of coverage probability of confidence intervals. The QLI method provided consistent estimates of trends for models with 60 or more cases, and this was true across variance patterns. We discuss QLI-generated estimates in relation to performance of OLS linear models. Finally, we present an example of how to apply the QLI method to quantify heteroscedasticity and how to interpret the estimates it provides, focusing on the implications for the OLS analysis. (PsycInfo Database Record (c) 2026 APA, all rights reserved).