Assessing and adjusting for bias in ecological analysis using multiple sample datasets.

Background: Ecological analysis utilizes group-level aggregate measures to investigate the complex relationships between individuals or groups and their environment. Despite its extensive applications across various disciplines, this approach remains susceptible to several biases, including ecological fallacy.

Methods: Our study identified another significant source of bias in ecological analysis when using multiple sample datasets, a common practice in fields such as public health and medical research. We show this bias is proportional to the sampling fraction used during data collection. We propose two adjustment methods to address this bias: one that directly accounts for the sampling fraction and another based on measurement error models. The effectiveness of these adjustments is evaluated through formal mathematical derivations, simulations, and empirical analysis using data from the 2014 Kenya Demographic and Health Survey.

Results: Our findings reveal that the sampling fraction bias can lead to significant underestimation of true relationships when using aggregate measures from multiple sample datasets. Both adjustment methods effectively mitigate this bias, with the measurement-error-adjusted estimator showing particular robustness in real-world applications. The results highlight the importance of accounting for sampling fraction bias in ecological analyses to ensure accurate inference.

Conclusion: Beyond the ecological fallacy uncovered by Robinson's seminar work, our research identified another critical bias in ecological analysis that is likely just as prevalent and consequential. The proposed adjustment methods provide potential tools for researchers to adjust for this bias, thereby improving the validity of ecological inferences. This study underscores the need for caution when pooling aggregate measures from multiple sample datasets and offers potential solutions to enhance the reliability of ecological analyses in various research domains.

Clinical trial number: Not applicable.