Not Cologne but the data collection (might have) changed everything: a cautionary tale on ignoring changes in data recording in sociological research

Abstract

Abstract In his 2020 ESR article, Arun Frey analysed data on ethnic violence that he web-scraped from a chronicle of hostile incidents against refugees, published by a German charitable foundation and a non-profit organization. He finds remarkable and supposedly causal effects of the New Year’s Eve sexual assaults in 2015/2016 on anti-refugee violence. We argue that it is invalid to draw conclusions regarding the effects of the New Year’s Eve event due to simultaneous alterations in the way the data have been collected. We review the changes in data reporting which took place at the beginning of 2016 on the level of the German government and discuss the lack of a remarkable New Year’s effect when focussing on data which has not been affected by these changes. This casts doubt on the previous findings proposed by Frey (2020). We conclude with a general word of caution.