Scoring the European citizen in the AI era

Abstract

Social scoring is one of the AI practices banned by the AI Act. This ban is explicitly inspired by China, which in 2014 announced its intention to set up a large-scale government project – the Social Credit System – aiming to rate every Chinese citizen according to their good behaviour, using digital technologies and AI. But in Europe, individuals are also scored by public and private bodies in a variety of contexts, such as assessing creditworthiness, monitoring employee productivity, detecting social fraud or terrorist risks, and so on. However, the AI Act does not intend to prohibit these types of scoring, as they would qualify as “high-risk AI systems”, which are authorised while subject to various requirements. One might therefore think that the ban on social scoring will have no practical effect on the scoring practices already in use in Europe, and that it is merely a vague safeguard in case an authoritarian power is tempted to set up such a system on European territory. Contrary to this view, this article argues that the ban has been drafted in a way that is flexible and therefore likely to make it a useful tool, similar and complementary to Article 22 of the General Data Protection Regulation, to protect individuals against certain forms of disproportionate use of AI-based scoring.