“Unprecedented Injustice”: network determinism in the Dutch digital welfare state

Diletta Huyskes
{"title":"“Unprecedented Injustice”: network determinism in the Dutch digital welfare state","authors":"Diletta Huyskes","doi":"10.1007/s43681-026-01350-6","DOIUrl":null,"url":null,"abstract":"<div><p>Automated decision-making systems in the welfare state have produced discriminatory outcomes at scale, as critical scholarship has extensively documented. This article examines the mechanisms through which such outcomes are produced by tracing the cultural, institutional, and technical determinants embedded in their design and use. Drawing on qualitative research conducted in the Netherlands between 2023 and 2024, it combines semi-structured interviews, ethnographic observation, and documentary analysis across three cases: SyRI, the childcare benefits scandal (Toeslagenaffaire), and Rotterdam’s welfare fraud detection system. Rather than asking whether automated systems discriminate, the article asks how discriminatory social classifications become translated into algorithmic risk and acquire the authority of seemingly objective decisions. The analysis identifies four recurring mechanisms: manual pre-selection of groups for scrutiny, the use of records of past suspicion as training data, proxy substitution for sensitive attributes, and the cumulative, intersectional layering of risk indicators. To conceptualize these dynamics, the article develops the notion of network determinism: the practical closure produced when administrative institutions, databases, policy objectives, statistical categories, organizational routines, and computational systems repeatedly reinforce one another, turning contingent classifications into effectively deterministic outcomes for those subjected to them. The findings show that discrimination cannot be attributed to algorithms alone, but emerges from chains of human and institutional choices that distribute both decision-making and responsibility. The Dutch cases therefore demonstrate why technical or legal corrections are insufficient without addressing the socio-cultural assumptions that automated systems are built to operationalize.</p></div>","PeriodicalId":72137,"journal":{"name":"AI and ethics","volume":"6 5","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2026-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43681-026-01350-6.pdf","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"AI and ethics","FirstCategoryId":"1085","ListUrlMain":"https://link.springer.com/article/10.1007/s43681-026-01350-6","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

Automated decision-making systems in the welfare state have produced discriminatory outcomes at scale, as critical scholarship has extensively documented. This article examines the mechanisms through which such outcomes are produced by tracing the cultural, institutional, and technical determinants embedded in their design and use. Drawing on qualitative research conducted in the Netherlands between 2023 and 2024, it combines semi-structured interviews, ethnographic observation, and documentary analysis across three cases: SyRI, the childcare benefits scandal (Toeslagenaffaire), and Rotterdam’s welfare fraud detection system. Rather than asking whether automated systems discriminate, the article asks how discriminatory social classifications become translated into algorithmic risk and acquire the authority of seemingly objective decisions. The analysis identifies four recurring mechanisms: manual pre-selection of groups for scrutiny, the use of records of past suspicion as training data, proxy substitution for sensitive attributes, and the cumulative, intersectional layering of risk indicators. To conceptualize these dynamics, the article develops the notion of network determinism: the practical closure produced when administrative institutions, databases, policy objectives, statistical categories, organizational routines, and computational systems repeatedly reinforce one another, turning contingent classifications into effectively deterministic outcomes for those subjected to them. The findings show that discrimination cannot be attributed to algorithms alone, but emerges from chains of human and institutional choices that distribute both decision-making and responsibility. The Dutch cases therefore demonstrate why technical or legal corrections are insufficient without addressing the socio-cultural assumptions that automated systems are built to operationalize.

“前所未有的不公”:荷兰数字福利国家的网络决定论
正如批判性学术广泛记录的那样,福利国家的自动化决策系统在规模上产生了歧视性结果。本文通过追踪嵌入其设计和使用中的文化、制度和技术决定因素,研究了产生这些结果的机制。它借鉴了2023年至2024年间在荷兰进行的定性研究,结合了半结构化访谈、人种学观察和对三个案例的文献分析:SyRI、托儿福利丑闻(Toeslagenaffaire)和鹿特丹福利欺诈检测系统。这篇文章没有问自动化系统是否存在歧视,而是问歧视性的社会分类如何转化为算法风险,并获得看似客观决策的权威。分析确定了四种反复出现的机制:人工预选审查群体,使用过去的怀疑记录作为训练数据,替代敏感属性的代理,以及风险指标的累积、交叉分层。为了将这些动态概念化,本文发展了网络决定论的概念:当行政机构、数据库、政策目标、统计类别、组织惯例和计算系统反复相互加强时,产生的实际封闭,将偶然分类转变为受其影响的有效确定性结果。研究结果表明,歧视不能仅仅归因于算法,而是源于分配决策和责任的人类和制度选择链。因此,荷兰的案例表明,如果不解决建立自动化系统的社会文化假设,技术或法律纠正是不够的。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书