Of hormones (well, not really!), behavior, and observer bias

In our laboratory, we measure hormones such as testosterone, melatonin, or corticosterone to relate hormone concentrations to the behavior of animals. Sometimes, we get samples of animals whose hormone concentrations had been altered by hormone implants or injections. Usually, those are measured to test if the hormonal manipulation had worked. I typically ask the experimenter which of the samples are from manipulated animals and which ones are from control animals. As an answer, I sometimes receive a raised eye brow and a questioning face. This kind of answer makes me happy because it shows my cooperation partner is aware of observer bias. I then explain that I would actually prefer not to know, which animals had been manipulated or not. However, our method to measure the hormones, the radioimmunoassay, requires me to know about it. The method is quite sensitive, but has a narrow range of concentrations in which we can reliably tell the correct hormone concentrations. Therefore, we need to adjust the dilution of the samples to remain in the expected range of concentrations we can measure with high precision. Hormone-treated samples might easily fall out of this range, if not diluted properly.

Presumably, the radioimmunoassay as a biochemical method is unlikely to produce observer bias, but this is different with behavioral observations, where our expectations as experimenters may inadvertently bias data collection. For this reason, good textbooks such as the classic Martin and Bateson (1985) or its latest edition (Bateson & Martin, 2021) caution against observer bias and also highlight the importance of testing for inter-observer reliability. A good way to do so is blinding observers to the treatment and apply established methods to test and improve inter-observer reliability.

About 12 years ago, Gordon Burghardt and colleagues investigated how major journals in animal behavior did with regard to reporting observer bias. They demonstrated that in 2010 major journals of our field (Animal Behaviour, Behaviour, Behavioural Ecology and Sociobiology, and Ethology) had observer bias reporting rates of <10%, therefore lagging behind Infancy, a journal on human infant behavior with reporting rates of more than 75% (Burghardt et al., 2012). Journal of Comparative Psychology had reporting rates of 20%.

In this issue of Ethology, Todd Freeberg, Scott Benson, and Gordon Burghardt offer a follow-up study (2024), showing that all behavioral journals have improved on reporting. This is good news! However, with rates in the range of 50% our field still lags behind Infancy, where basically every study reports observer bias and tests for inter-observer reliability. Also, Journal of Comparative Psychology still does better with roughly 75% or articles reporting on observer bias. Admittedly, it is more difficult and sometimes impossible to conduct blind observations in animal behavior studies, especially when data are collected in the field, but for sure there is still scope to improve blinding and reporting. Behavioural Ecology and Sociobiology does slightly better than other journals including Ethology, possibly because they have implemented a submission policy requiring authors to report on observer bias. As a response to Freeberg et al. (2024) renewed investigation Ethology now also decided to require authors to report how they dealt with observer bias. Hopefully, in few more years, Freeberg and colleagues can report that we left infancy behind and our journals reached a mature state of dealing with observer bias.

Wolfgang Goymann: Conceptualization; writing – original draft; writing – review and editing.