Ethical scaling for content moderation: Extreme speech and the (in)significance of artificial intelligence

IF 6.5 1区社会学 Q1 SOCIAL SCIENCES, INTERDISCIPLINARY

Big Data & Society Pub Date : 2023-01-01 DOI:10.1177/20539517231172424

Sahana Udupa, Antonis Maronikolakis, Axel Wisiorek

{"title":"Ethical scaling for content moderation: Extreme speech and the (in)significance of artificial intelligence","authors":"Sahana Udupa, Antonis Maronikolakis, Axel Wisiorek","doi":"10.1177/20539517231172424","DOIUrl":null,"url":null,"abstract":"In this article, we present new empirical evidence to demonstrate the severe limitations of existing machine learning content moderation methods to keep pace with, let alone stay ahead of, hateful language online. Building on the collaborative coding project “AI4Dignity” we outline the ambiguities and complexities of annotating problematic text in AI-assisted moderation systems. We diagnose the shortcomings of the content moderation and natural language processing approach as emerging from a broader epistemological trapping wrapped in the liberal-modern idea of “the human”. Presenting a decolonial critique of the “human vs machine” conundrum and drawing attention to the structuring effects of coloniality on extreme speech, we propose “ethical scaling” to highlight moderation process as political praxis. As a normative framework for platform governance, ethical scaling calls for a transparent, reflexive, and replicable process of iteration for content moderation with community participation and global parity, which should evolve in conjunction with addressing algorithmic amplification of divisive content and resource allocation for content moderation.","PeriodicalId":47834,"journal":{"name":"Big Data & Society","volume":" ","pages":""},"PeriodicalIF":6.5000,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"3","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Big Data & Society","FirstCategoryId":"90","ListUrlMain":"https://doi.org/10.1177/20539517231172424","RegionNum":1,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"SOCIAL SCIENCES, INTERDISCIPLINARY","Score":null,"Total":0}

引用次数: 3

Abstract

In this article, we present new empirical evidence to demonstrate the severe limitations of existing machine learning content moderation methods to keep pace with, let alone stay ahead of, hateful language online. Building on the collaborative coding project “AI4Dignity” we outline the ambiguities and complexities of annotating problematic text in AI-assisted moderation systems. We diagnose the shortcomings of the content moderation and natural language processing approach as emerging from a broader epistemological trapping wrapped in the liberal-modern idea of “the human”. Presenting a decolonial critique of the “human vs machine” conundrum and drawing attention to the structuring effects of coloniality on extreme speech, we propose “ethical scaling” to highlight moderation process as political praxis. As a normative framework for platform governance, ethical scaling calls for a transparent, reflexive, and replicable process of iteration for content moderation with community participation and global parity, which should evolve in conjunction with addressing algorithmic amplification of divisive content and resource allocation for content moderation.

查看原文本刊更多论文

内容节制的伦理尺度：极端言论和人工智能的意义

在这篇文章中，我们提出了新的经验证据，以证明现有的机器学习内容调节方法在跟上，更不用说领先于网络仇恨语言方面的严重局限性。在协作编码项目“AI4Dimity”的基础上，我们概述了在人工智能辅助审核系统中注释问题文本的模糊性和复杂性。我们将内容节制和自然语言处理方法的缺点诊断为从自由主义现代“人”思想中包裹的更广泛的认识论陷阱中出现。提出了对“人与机器”难题的非殖民化批判，并提请人们注意殖民主义对极端言论的结构性影响，我们提出了“道德尺度”，以强调温和过程是政治实践。作为平台治理的一个规范框架，道德扩展需要一个透明、反射性和可复制的迭代过程，以实现社区参与和全球平等的内容审核，这应该与解决分裂性内容的算法放大和内容审核的资源分配相结合。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Big Data & Society SOCIAL SCIENCES, INTERDISCIPLINARY-

CiteScore

10.90

自引率

10.60%

发文量

审稿时长

11 weeks

期刊介绍： Big Data & Society (BD&S) is an open access, peer-reviewed scholarly journal that publishes interdisciplinary work principally in the social sciences, humanities, and computing and their intersections with the arts and natural sciences. The journal focuses on the implications of Big Data for societies and aims to connect debates about Big Data practices and their effects on various sectors such as academia, social life, industry, business, and government. BD&S considers Big Data as an emerging field of practices, not solely defined by but generative of unique data qualities such as high volume, granularity, data linking, and mining. The journal pays attention to digital content generated both online and offline, encompassing social media, search engines, closed networks (e.g., commercial or government transactions), and open networks like digital archives, open government, and crowdsourced data. Rather than providing a fixed definition of Big Data, BD&S encourages interdisciplinary inquiries, debates, and studies on various topics and themes related to Big Data practices. BD&S seeks contributions that analyze Big Data practices, involve empirical engagements and experiments with innovative methods, and reflect on the consequences of these practices for the representation, realization, and governance of societies. As a digital-only journal, BD&S's platform can accommodate multimedia formats such as complex images, dynamic visualizations, videos, and audio content. The contents of the journal encompass peer-reviewed research articles, colloquia, bookcasts, think pieces, state-of-the-art methods, and work by early career researchers.