Learning your individual inferences: A guide for overcoming statistical challenges in small-N studies

Selecting an appropriate statistical test can be challenging for animal scientists. This is particularly true for those who study animals in applied animal settings, where a small number of subjects studied (i.e., small-N) is commonplace. Small-N studies regularly coincide with additional problems, such as non-normally distributed data and repeated measures, making many of the standard independent samples-based inferential statistics less appropriate. Some researchers may persist in using these tests irrespective of assumption violations, and in doing so they risk a Type 1 (false positive) error, potentially leading to erroneous conclusions about their data. Other, lesser considered challenges such as a lack of homogeneity of variance between conditions and time-dependency, are also commonly encountered in small-N studies. If not considered, these challenges could result in extra noise in a dataset that could reduce reliability of results. Fortunately, alternative tests are available that can account for these issues, including issues of non-independence, such as paired data and time-dependency. This guide provides simulated data to generate scenarios that reflect actual problems that emerge in small-N research. Using these generated datasets, a series of tests are used to demonstrate how they can overcome some of the statistical noise encountered. Our goal is to provide researchers with an outline of assumptions and appropriate tests to help them overcome commonly faced challenges for small-N studies.