On Frequency and Probability Weights: An In‐Depth Look at Duelling Weights

SummaryProbability weights have been widely used in addressing selection bias arising from a variety of contexts. Common examples of probability weights include sampling weights, missing data weights, and propensity score weights. Frequency weights, which are used to control for varying variabilities of aggregated outcomes, are both conceptually and analytically different from probability weights. Popular software such as R, SAS and STATA support both types of weights. Many users, including professional statisticians, become bewildered when they see identical estimates, but different standard errors and ‐values when probability weights are treated as frequency weights. Some even completely ignore the difference between the two types of weights and treat them as the same. Although a large body of literature exists on each type of weights, we have found little, if any, discussion that provides head‐to‐head comparisons of the two types of weights and associated inference methods. In this paper, we unveil the conceptual and analytic differences between the two types of weights within the context of parametric and semi‐parametric generalised linear models (GLM) and discuss valid inference for each type of weights. To the best of our knowledge, this is the first paper that looks into such differences by identifying the conditions under which the two types of weights can be treated the same analytically and providing clear guidance on the appropriate statistical models and inference procedures for each type of weights. We illustrate these considerations using real study data.