内容下架和活动家组织:社交媒体内容审核对活动家和组织的影响

IF 4.1 1区 文学 Q1 COMMUNICATION
Diane Jackson
{"title":"内容下架和活动家组织:社交媒体内容审核对活动家和组织的影响","authors":"Diane Jackson","doi":"10.1002/poi3.372","DOIUrl":null,"url":null,"abstract":"Social media companies are increasingly transcending the offline sphere by shaping online discourse that has direct effects on offline outcomes. Recent polls have shown that as many as 70% of young people in the United States have used social media for information about political elections (Booth et al., 2020) and almost 30% of US adults have used social media to post about political and social issues (McClain, 2021). Further, social media have become a site of organizing with over half of US adults reporting having used social media as a tool for gathering or sharing information about political or social issues (Anderson et al., 2018). Despite the necessity of removing content that may breach the content guidelines set forth by social media companies such as Facebook, Instagram, Twitter, and TikTok, a gap persists between the content that violates guidelines and the content that is removed from social media sites. For activists particularly, content suppression is not only a matter of censorship at the individual level. During a time of significant mobilization, activists rely on their social media platforms perhaps more than ever before. This has been demonstrated by the Facebook Arabic page, “We Are All Khaled Said,” which has been credited with promoting the 2011 Egyptian Revolution (Alaimo, 2015). Activists posting about the Mahsa Amini protests and Ukrainians posting about the Russian invasion of Ukraine have reported similar experiences with recent Meta policy changes that have led to mass takedowns of protest footage and related content (Alimardani, 2022). The impacts of social media platforms' policy and practices for moderation are growing as cyberactivism has become more integral in social organizing (Cammaerts, 2015). However, due to accuracy and bias issues of content moderation algorithms deployed on these platforms (Binns et al., 2017; Buolamwini & Gebru, 2018; Rauchberg, 2022), engaging social media as a tool for social and political organizing is becoming more challenging. The intricacies and the downstream systemic effects of these content moderation techniques are not explicitly accounted for by social media platforms. Therefore, content moderation is pertinent to social media users based on the effects that moderation guidelines have not only on online behavior but also on offline behavior. The objectives of this paper are twofold. First and foremost, the goal of this paper is to contribute to the academic discourse raising awareness about how individuals are being silenced by content moderation algorithms on social media. This paper does this primarily by exploring the social and political implications of social media content moderation by framing them through the lens of activism and activist efforts online and offline. The secondary goal of this paper is to make a case for social media companies to develop features for individuals who are wrongfully marginalized on their platforms to be notified about and to appeal incidences of censorship and content removal. To do so, this paper will begin by discussing common techniques of content moderation including algorithmic content moderation, which various social media platforms have used to moderate content on their sites (Morrow et al., 2022). This paper will then address the impacts of algorithmic content moderation on its users generally, on individuals identifying with marginalized communities, and on activists by reviewing recent literature in this area and by citing recent examples of social movements impacted by social media censorship. Finally, this paper will consider the downstream systemic implications of algorithmic content moderation both politically and ethically for both social media users and members of society at large. Social media have evolved tremendously since their inception. With it, content moderation practices have also evolved (Morrow et al., 2022). However, social media companies, users, and regulators have different and sometimes competing motivations when it comes to content moderation (Morrow et al., 2022). As such, these practices have been criticized as inconsistent and lacking transparency (Ananny & Gillespie, 2016; Morrow et al., 2022) as they tend to produce significant errors that directly impact users (West, 2018). Further, these practices are seemingly not growing at a rate necessary to effectively and accurately accomplish the tasks for which it was developed while simultaneously expanding to reach the scale at which social media platforms are expanding. Although social media companies have not consistently or transparently moderated the content on their platforms or disclosed their moderation techniques (Bone, 2021; Morrow et al., 2022), mainstream social media platforms have tended to integrate an algorithmic content moderation technique derived from a manual coding process (Morrow et al., 2022). One such moderation approach involves moderating specific tags and keywords that may be damaging or inappropriate and is referred to as content labeling (see Gerrard, 2018). However, this approach has its drawbacks that make it impossible to function on its own. One recent study looked at proeating disorder content on image-sharing sites and online communities and found that users found ways of circumventing the hashtag and other forms of policing on the sites, emphasizing the limitations of the approach (see Gerrard, 2018). Though evidence of this still exists, it cannot sufficiently function alone. Other approaches incorporate more machine learning techniques and advanced pattern matching that compares new content to existing examples of training data developed by human coders and branded under the broad umbrella of artificial intelligence (AI; see Gillespie, 2020; Gorwa et al., 2020; Langvardt, 2018). This style of approach involves developers grouping together posts or comments as a sample of training data, which human coders manually code (Gillespie, 2020). At this point, the human coders feed the training data with its codes into a program, which uses machine learning techniques to extrapolate from the patterns of the coded training data to match other content in the social media platform (Gillespie, 2020). This format can be applied to both written content in social media posts, comments, and captions as well as to images or videos. The outcome of these approaches, however, seems to involve only presenting selected information and inappropriately omitting or hiding other information (Marcondes et al., 2021). Further, because this approach also incorporates features for users to flag or report offensive or inappropriate content, humans are still involved in the flagging of content by users and, sometimes, in the decision to remove the content (Gillespie, 2020). This paper will center around algorithmic content moderation generally and will weigh the benefits and drawbacks of these approaches before considering how they directly impact marginalized individuals and activist efforts. Algorithmic content moderation has two clear benefits: its reach and cost-effectiveness relative to compensating numerous human moderators who would be tasked with pouring over and flagging every offensive or inappropriate piece of content. The ever-growing size of data and simplistic judgments required to moderate content tend to be justifications that mainstream social media platforms use for the implementation of AI for content moderation (Gillespie, 2020; Gorwa et al., 2020). According to these companies, only AI can address the problems posed by the data, judgments, and user and content amount (Gillespie, 2020). While AI may address the issue of size (Gorwa et al., 2020) and may seem like an economical solution to protect human moderators from consuming damaging, hateful content and to preserve their time from making judgments that could be made by a computer (Gillespie, 2018, 2020), pervasive problems exist on these platforms that seem to mirror sociopolitical problems offline (Gorwa et al., 2020). Although algorithmic content moderation has been framed as the solution to many of the moderation issues that mainstream social media companies currently face and as an answer to legislative calls to address security and safety concerns on these platforms (Gorwa et al., 2020), this approach may present more entrenched problems than it addresses. Many of the existing content moderation algorithms discussed in recent literature seem to only engage AI in a broader sense, where much of the technology follows advanced methods of pattern matching of content first identified by human manual coders (Gorwa et al., 2020). In this sense, this technology is still dependent on humans to make initial judgments from which these tools follow inductive logic to flag similar or duplicate content instead of making original judgments itself. As such, the capability of these tools to identify inappropriate content stagnates rather than evolves over time with the trends of human language and interaction. Further, machine learning tools necessitate training data as indicators of classifying particular instances into specific categories (Duarte et al., 2017). The issue with the architecture of these tools is that these training data are necessary for the algorithms to identify similar or the same instances. However, the algorithms are fixed, then, to the similar or same instances of the training cases provided (Sinnreich, 2018). As such, the capacity for machine learning tools to account for cultural context, humor, subtlety, and other nuances that human judgment can identify is limited and fixed to the diversity of the messages captured by the training data it was provided (Duarte et al., 2017). This is particularly problematic due to the lack of representativeness that training data may have, the issues that these models tend to have with performing consistently across different domains, and the unrealistically rigid definitions that these tools apply when processing language (Duarte et al., 2017; Sinnreich, 2018). Lastly, because algorithmic content moderation is designed to remove content or hide it altogether, the lack of accuracy to correctly classify content is particularly damaging (Duarte et al., 2017), both for those who are sharing content that does not adhere with platform guidelines (Gerrard, 2018) and for those who are sharing guideline-adherent content (Marcondes et al., 2021; West, 2018). This is partly due to the lack of feasibility and ease with which users can circumvent platform moderation techniques. This was demonstrated by a case study documented by Gerrard (2018) of proeating disorder content shared by individuals whose content was not identified as damaging or harmful by the moderation algorithms of several mainstream photo-sharing and community-hosting social media sites. However, those whose content is being wrongly restricted or removed altogether are paying a price (Alimardani, 2022; Marcondes et al., 2021; Middlebrook, 2020; Morrow et al., 2022; West, 2018). As has been discussed in this section, these algorithmic moderation practices are only as accurate and unbiased as the individuals who have developed these practices and as up-to-date as the most recent update to these practices (Binns et al., 2017; Gorwa et al., 2020). As such, these practices are left open to the biased and systematic over-policing of certain groups and the neglect of others (Marcondes et al., 2021; Morrow et al., 2022; Sinnreich, 2018), which thus continually reinforces offline hegemonic principles (Middlebrook, 2020). This topic will be covered more in the following section. Machine learning tools rely on identifying and extracting patterns of behavior and responses from instances among groups of text and users. In doing so, the structure of machine learning necessitates the normalization of hegemonic principles and practices, which thus, can lead to the inaccurate and ineffective identification of these patterns among marginalized communities. As discussed by Gillespie (2020, p. 3), “the margin of error typically lands on the marginal: who these tools over-identify, or fail to protect, is rarely random.” Marginalization is defined in this paper as “both a condition and a process that prevents individuals and groups from full participation in social, economic, and political life enjoyed by the wider society” (Alakhunova et al., 2015, p. 10). Multiple recent analyses have been published about social media censorship and taking special note of the censorship of individuals from marginalized communities (see Alimardani, 2022; Dinar, 2021; Duarte et al., 2017; Knight Foundation & Gallup Inc., 2020; Smith et al., 2021). Specifically, analyses like these identify the key challenges, takeaways, and implications of the systemic censorship of individuals belonging to marginalized communities. One form of censorship identified broadly by these papers and other recent research articles is that of shadowbanning, or the algorithmic censorship, hiding, or removal of users' content who are said to have violated community guidelines and which results in many marginalized users' public content being hidden from feeds and searches without their awareness (Knight Foundation & Gallup Inc., 2020; Rauchberg, 2022; Smith et al., 2021). Twitter (Twitter Help Center, 2023) and Instagram (Instagram Help Center, 2023) both publicize on their sites the practice of making content less visible on more public locations such as hashtags or Explore pages or when searched. Due to the lack of a notification process for those whose content may be censored in this way, individuals are not given the opportunity to be made aware of their content or profiles being censored or how long the censorship will last. In a recent survey, almost every marginalized group reported experiencing censorship on Instagram at disproportionally higher percentages than more privileged groups (Smith et al., 2021). Specifically, this survey reported that sex workers, sex educators, LGBQIA+, trans, nonbinary, black, indigenous, and people of color, and disabled individuals all reported experiencing content removals from Instagram at higher rates than those in more privileged groups. Leaked internal documents from TikTok “instructed moderators to suppress posts created by users deemed too ugly, poor, or disabled for the platform” (Biddle et al., 2020, para. 1) and to censor political discourse during livestreams to attract new users. Further, TikTok has admitted to bias in its antibullying moderation system, which suppressed the content of individuals who appeared to have a disability, facial disfigurement, or other characteristics that may make them vulnerable to being bullied (Botella, 2019; Köver & Reuter, 2019; Rauchberg, 2022). As such, the app's brief history has been riddled with evidence of the algorithmic suppression of certain groups. Other leaked information about the company revealed the tactics that TikTok's parent company, ByteDance, has used to moderate, suppress, or censor content that goes against China's foreign policy goals (Hern, 2019). This evidence shows a moderation of content about Tiananmen Square and Tibetan independence (Hern, 2019) and substantiates accusations against the company for censoring content about the Hong Kong protests (Harwell & Romm, 2019). Another research article about shadowbanning marginalized individuals underscores how more vulnerable communities are being disproportionately targeted by these moderation techniques on- and offline via Instagram (Middlebrook, 2020). This piece considered the implications that marginalized communities face being shadowbanned given the lack of awareness that users generally have about moderation algorithms. Unfortunately, the lack of guidelines provided to users to know when or why they are being shadowbanned and the lack of an appeals process to reverse the shadowban make this a particularly unjust penalization. Censorship and surveillance adversely affect users in marginalized communities and activists (Asher-Schapiro, 2022) who rely on social media to organize (Alimardani, 2022; Sandoval-Almazan & Ramon Gil-Garcia, 2014). For activists and individuals in marginalized communities, this online oppression can be the difference-maker for their organizing efforts and information sharing to be successful. The lack of specificity that social media companies offer around these practices and the lack of accuracy that social media content moderation algorithms can have when moderating the content of marginalized individuals has major implications on users' freedom of speech. The lack of transparency into moderation algorithms and recent changes to social media sites (e.g., Elon Musk purchasing Twitter; Asher-Schapiro, 2022) have prompted more discourse regarding the way social media sites moderate conversations (see Gerrard, 2018) and who are actually being moderated (Middlebrook, 2020). These systemic inadequacies of moderation algorithms are exacerbated by the common practices of most sites like these to depend on their users to report content as offensive or inappropriate instead of detecting the content through other methods (Chen et al., 2012). Relying on and entrusting users to report content becomes a greater issue of suppressing marginalized, guideline-adherent content. Users could exploit their privileges to report content for removal regardless of whether the content adheres to the site's community guidelines. Existing research about partisan-motivated behaviors like distributing fake news articles (Osmundsen et al., 2021) and labeling news as fake when it disconfirms partisans' existing beliefs (Kahne & Bowyer, 2017) offers support for the possibility that social media users may use features to report content for removal when it contradicts their attitudes. As such, activists may be up against a double bind where the automated content moderation systems in place may unfairly and inaccurately remove their content automatically and where individuals who disagree with them may flag their content to these inequitable automated systems for removal. This double bind puts activists who use social media at a disadvantage in their efforts to share their messages, despite social media becoming increasingly more necessary to engage in activism (Cammaerts, 2015). This section will discuss two recent and important examples of social movements that used social media in unique and important ways and the effect that content takedowns had or could have had on the movements. The BLM movement originated with the social media hashtag #BlackLivesMatter after George Zimmerman, the man who shot and killed Trayvon Martin in 2012, was acquitted. The hashtag and movement gained tremendous momentum in the United States in 2014 after Michael Brown in Missouri and Eric Garner in New York were killed in incidents of police brutality (Howard University School of Law, 2023). The hashtag and movement spread globally after the police murder of George Floyd in Minnesota in 2020 (Howard University School of Law, 2023; Tilly et al., 2019). In a study about collegiate athlete activism in the United States, collegiate athlete activists considered social media as one of the primary modalities through which activism occurs and through which activists can reach, amplify, and engage with messages from other members of their movements (Feder et al., 2023). For some, demonstrations took place exclusively on social media and for others, social media was an integral component of amplifying the calls to action that they publicized. These activists' organizing efforts manifested in players' strikes, donation drives after riots in the wake of the murder of George Floyd, voting campaigns, and even sales of BLM masks, the proceeds from which were donated to charities geared toward helping marginalized communities (Feder et al., 2023). Censoring the messages of these activists during their organizing efforts and during attempts to mobilize becomes an issue of political freedom as well as free speech. These experiences of suppressing political freedom and free speech were voiced by BLM activists who used TikTok during the height of BLM demonstrations that came about after the murder of George Floyd (McCluskey, 2020). These activists described drastic reductions of appearances that their videos had on users' “For You” pages on TikTok (McCluskey, 2020; Nicholas, 2022). After becoming a central site for sharing activism content (Janfaza, 2020), TikTok released a statement apologizing to its community of Black creators and users who had previously voiced feelings of marginalization and suppression on the app (see Pappas & Chikumbu, 2020). According to the official statement released by TikTok, Black content creators and allies who are users of the app changed their profile pictures and spoke out through the platform to discuss their experiences of marginalization on TikTok (Pappas & Chikumbu, 2020). The statement went on to explain that “a technical glitch made it temporarily appear as if posts uploaded using #BlackLivesMatter and #GeorgeFloyd would receive 0 views” (para. 3). Unfortunately, almost 2 months after the release of this statement, Black TikTok creators voiced continued experiences of content suppression and marginalization on the app (McCluskey, 2020). BLM activists who use the app reported not only lacking visibility to other #BlackLivesMatter content on their personalized feeds, but racist and offensive comments shared by individuals who were shown their BLM content (Allyn, 2020). Black creators who returned to posting non-BLM content on TikTok found that their regular content was receiving far less engagement (McCluskey, 2020). Further, one Black creator, Emily Barbour, recounted to TIME Magazine having created a post from a screen recording that she took containing another TikTok video in which the creator appeared in blackface in an effort to call out the video (McCluskey, 2020). While Barbour's video was hidden for being deemed a copyright violation, TikTok did not consider the video featuring a person in blackface as violating its community guidelines, leaving the video up for two more days despite thousands of users having flagged the video (McCluskey, 2020). Barbour shared with TIME the experience of racial bias and unfair treatment on TikTok (McCluskey, 2020). TikTok is not the only social media platform with documented instances of suppressing content from BLM activists. Louiza Doran is a antiracism activist and educator who experienced censorship on Instagram (Silverman, 2020). According to Doran in an interview with BuzzFeed News, her account was prohibited from livestreaming and some of her posts and comments were removed altogether (Silverman, 2020). Having been described by a spokesperson from Facebook as a technical issue that resulted in Doran's Facebook and Instagram accounts being flagged as spam (Silverman, 2020), the explanation of this issue bears striking resemblance to TikTok's “technical glitch” that caused BLM-related posts to be displayed differently (Pappas & Chikumbu, 2020). This common thread of social media companies blaming technical bugs as reasons for suppressing and censoring content points to: (1) the shortcomings of algorithmic content moderation, (2) the lack of visibility that social media companies have into the downstream effects of these moderation systems, and (3) the scapegoating strategy that these platforms use to dismiss concerns of systemic moderation bias as technological difficulty. For activists and marginalized individuals, these are not just setbacks but critical obstacles that encumber not only these people, but their followers, and, thus, factions of entire movements. The Mahsa Amini protests in Iran, which have sparked the largest antiregime uprising the country has seen since the Iranian Revolution in 1979 (France-Presse, 2022), have relied on features like hashtags and encryption on platforms like Instagram and Twitter to spread awareness and amplify messages of the protests within and outside of Iran (Amidi, 2022; Kenyon, 2023; Kumar, 2022). These antiregime protests began in 2022 in Iran and have continued as of September 2023 (Kenyon, 2023), and messages of this movement have spread across the world. Activists have been forced to rapidly transition between social media sites to counteract the Iranian government's efforts to suppress the movement's messages (Amidi, 2022; Iran International, 2022; Kumar, 2022). Due to the Iranian government's response to suppress the movement and its actors with violence, censorship of social media posts, and internet outages (CNN Special Report, 2023; France-Presse, 2022), Iranian protesters have had to depend on individuals outside of the country as well as innovations like encryption and offsite servers to share protest information on social media and to stay up-to-date (Amidi, 2022; Butcher, 2022; Kumar, 2022). Amidst the governmental censorship and violence against protestors (Amidi, 2022; CNN Special Report, 2023), the movement and its members have demonstrated resilience by quickly pivoting between social media platforms and by amplifying its messages through their online and offline presence (Kumar, 2022). As such, social media has played a critical role in the movement's sustained impact (Amidi, 2022; Kumar, 2022). Specifically, Instagram has become one of Iran's most popular social media platforms and it is the only uncensored foreign form of social media (Alimardani, 2022; Dagres, 2021). However, a policy change that Meta made in 2022 in response to the Russian invasion of Ukraine has led to the removal of large amounts of Iranian protest-related content that contained a common Iranian protest slogan (Alimardani, 2022). This slogan, which translates into English as calls for death to the military forces, current supreme leader, and current president, exist culturally as a symbolic dissident call against Iran's authoritarian regime rather than a true call for violence (Alimardani, 2022). Where Meta used to include these slogans in their exceptions of community guidelines, a new policy change that Meta made rolls back these exceptions both for these slogans in Iran and for other protest slogans used in Ukraine during the Russian invasion (Alimardani, 2022; Biddle, 2022; Vengattil & Culliford, 2022). As such, Meta has been criticized for prioritizing more respect for the dictators than for the protestors (Höppner, 2022) where their policies regarding human rights and free speech more align with U.S. policy (Biddle, 2022). For Iranian protesters, their government's capabilities paired with the policy decisions that social media companies like Meta make in moderating their content pose a unique combined challenge for organizing and mobilizing. This mass systematic removal of posts from personal accounts and important national media outlets that came about without warning in the wake of Meta's policy change (Alimardani, 2022) has exacerbated the extent to which these protesters are forced to monitor their posts and strategize their technology use. Further, evidence of the Iranian government hacking activists' accounts or taking control of the accounts of protesters in their custody (Polglase & Mezzofiore, 2022) pose significant threats to activists and to their movement. Indeed, social media platforms' moderation policies and practices have important implications on activists and their movements. This Meta content moderation policy change has produced a similar outcome to the censorship practiced by the Iranian government. As such, this situation speaks to the necessity for social media platforms to identify and institute culturally nuanced understanding in their moderation policies and practices if their objectives truly involve promoting free speech on their sites. In addition to the concerns that algorithmic moderation systems perpetuate systemic political injustices that persist offline, these practices come with numerous ethical concerns and issues of scale (Gorwa et al., 2020; Marcondes et al., 2021). Therefore, the following sections will review political and ethical concerns and recommendations specifically as these algorithmic moderation practices pertain to activism and organizing. There are many political implications of the current structure of algorithmic content moderation that should be considered. Because social media companies' content moderation algorithms' display of bias against marginalized individuals has been documented (Middlebrook, 2020; Rauchberg, 2022) and because activists are often individuals who identify with at least one marginalized group (Heaney, 2022; Kluch, 2022), erroneously removing content shared by individuals who are activists and who are marginalized exacerbates the marginalization (Middlebrook, 2020). Societally, this practice stunts movements, hampers representation, targets groups, silences individuals, violates rights to free speech, and multiplies the workload of the content creators who identify in these ways and experience this mistreatment. In addition to the specific issues that are considered in activists' posts, these individual implications carry weight for political engagement, political information seeking, and public policy. Assumptions of western ideals such as freedom of speech and freedom of the press underlie the discussion in this paper. As polarization across party lines on sociopolitical issues continues to grow, partisan behaviors such as sharing news articles that disparage opposing political parties and party members on social media have persisted (Osmundsen et al., 2021). Of course, reporting content is now feasibly emerging as another possible partisan behavior that works to disparage opposing perspectives by flagging them for removal. However, these behaviors are not only centralized around western countries where apparent government censorship may not be present because censorship may come from sources other than government entities (e.g., host social media companies or third parties). For activists in countries where government censorship of social media content is exerted (see Casilli & Tubaro, 2012; Golovchenko, 2022; Liu & Zhao, 2021; Poell, 2014; Tai & Fu, 2020), activists' labor is multiplied to both create and publicize their messages, mobilize individuals, and replicate this process to combat either the algorithmic content removal or governmental censorship. Because the individuals who create and deploy algorithms are embedded in systems of oppression, their products reproduce these systems (Middlebrook, 2020), which ultimately oppresses activists and marginalized individuals, thereby oppressing social movements and organizing processes, political discourse, political engagement, and, thus, political outcomes. As such, even activists who reside in countries that tout the prioritization of free speech are experiencing instances of censorship (Langvardt, 2018). Instances like these where certain perspectives are excluded from discourse have been shown to impact others' perceptions of public opinion and marginalize the expression of those with similar positions (Gearhart & Zhang, 2015; Roy, 2017). Ethical concerns for algorithmic content moderation practices involve democratizing content removal features, the relational and emotional ramifications of these systems, and the lack of cultural and linguistic understanding that these practices demonstrate. Enabling all users to report content for removal broadens the potential for innocent content to be removed automatically by these moderation algorithms as it puts the power in users' hands, thereby increasing the amount of flagged content that is appropriate and, thus, the number of ambiguous cases that human moderators would potentially need to review. Though social media companies have focused on effectively reducing manual moderation while keeping hate speech and cyberbullying at bay, the overreach of these algorithms relative to the inaccuracy of these moderation algorithms has not been considered (Center for Countering Digital Hate, 2022). However, existing research has also documented relational and emotional ramifications that individuals who report having been censored and shadowbanned experience online and offline (West, 2018). What many may not also recognize is the emotional toll that is placed on both human moderators who work for social media companies (Gillespie, 2018, 2020) and those who consume content that should be removed and was not detected by algorithms (Center for Countering Digital Hate, 2022). It is critical to recognize the emotional labor that human content moderators tasked with categorizing content into appropriate content and inappropriate content in the building of training datasets are confronted with (Gillespie, 2018; Sinnreich, 2018). Further, although human content moderators can identify ironic statements and greater nuance than moderation algorithms (Duarte et al., 2017), human content moderation applies the same level of bias to what is considered hate speech or inappropriate content with potentially more inconsistency. Given the socially constructed nature of hate speech and what it is or is not, the notion of hate speech may differ culturally and cannot be fixed to a singular point in time (Gillespie, 2020). Meta's content moderation policy change that led to the mass removal of posts using Iran's dissident slogan is an important example of this (Alimardani, 2022). Meta has also demonstrated an incapacity for moderating content in non-Western languages and applying Western ideals to moderation policies in other parts of the world (Alimardani & Elswah, 2021; Fatafta, 2022). Thus, Meta has demonstrated a lack of cultural sensitivity in its creation and institution of moderation policy and practices. These issues, however, may be ameliorated by using moderation algorithms to flag the most egregious and universally held forms of offensive speech and promoting more ambiguous posts to human coders (Gillespie, 2020). This change and increased transparency of social media companies about the moderation algorithms they deploy and content removal for users are meaningful developments that social media platforms could offer to help wrongfully censored content be corrected. Although each of these implications are significant, issues such as the lack of transparency into the innerworkings of social media moderation algorithms and the lack of structural opportunities to appeal or inquire about moderation practices are some of the most integral. Despite the fact that experts in AI have developed ethics guidelines for users and for developers (see Ryan & Stahl [2020] for one example), social media platforms such as Instagram or Twitter do not seem to publicize or practice ethical competencies that guide their algorithm development and implementation. In fact, with Elon Musk's purchase of Twitter, the team dedicated to ethically practicing AI and promoting transparency and fairness was reportedly disbanded (Knight, 2022). Social media companies should also broaden their organizational structure so that developers institute moderation practices specific to the regions and languages of which they demonstrate sufficient cultural understanding. By offering more transparent affordances such as notifications and appeals features for users whose content is removed or censored from searches, users can be made aware of how their content is violating regulations and appeal these decisions. Because Twitter, Instagram, and Facebook have notifications and appeals options in place for other instances, these companies already have the infrastructure that could be extended to allow for these possibilities. Further, algorithms could be updated as coders review appeals in an effort to protect and better represent the groups of people like activists and marginalized individuals whose content tends to be removed more often and unjustly. Additional features could involve implementing a business profile category for users to label their accounts as political activists. Social media companies could then vet the profiles and institute exemptions in their content moderation algorithms specifically for accounts with this profile category type. The first goal of this paper was to contribute to the academic discourse sharing more information about how certain individuals are being censored and silenced by social content moderation algorithms. To do this, this paper considered the various forms of social media content moderation along with how these features work in tension with free speech and equal treatment of user groups that are espoused by these platforms. The second goal of this paper was to propose for social media companies to implement greater transparency into their content moderation practices. This paper recommends extending their current appeals infrastructure to notify users when their content or accounts are being hidden from searches or public pages, changing the structure of algorithmic content moderation, and offering additional features to make moderation more accurate. By implementing algorithms for more extreme, offensive content, enlisting human moderators to code more ambiguous, nuanced posts (Gillespie, 2020) and building algorithms to be more iterative so that they can be updated as coders review appeals to automated moderation decisions, moderation algorithms will have a much greater capacity for ethically, accurately, and effectively moderating hate speech while promoting free speech equally. Social media companies have a way to go before arriving at a finish line for ethically, accurately, and effectively mitigating hate speech and encouraging political and social discourse. Major recurring issues such as the lack of transparency of the innerworkings of social media algorithmic moderation systems and the lack of structural opportunities to appeal or inquire about moderation practices are only the tip of the iceberg. As marginalized communities are targeted through automated moderation practices like shadowbanning, oppressive and inequitable treatment of these users is reinforced online while offline hegemonic systems that silence marginalized communities grow stronger. The issue of social media content moderation—particularly algorithmic content takedowns—and how they affect social activists who are using social media as a tool for organizing is of significance both for social justice movements and their members as well as social media platforms and their stakeholders. As cyberactivism and the online amplification of offline social issues integrate with physical mobilizing (Sandoval-Almazan & Ramon Gil-Garcia, 2014), the role of social media in supporting social activists' online communication is increasingly important. Equipping all users, regardless of the extreme ideological positions that they may carry or vocalize, with these tools further exacerbates the gap between offensive or inappropriate content and removed content. Doing so runs counter to the notion of free speech, thereby making this capability an issue of public policy. Indeed, making features like this widely accessible does more to reinforce the offline hegemony online and contributes more to engaging in free content flagging than it does to engaging in free speech.","PeriodicalId":46894,"journal":{"name":"Policy and Internet","volume":"3 3","pages":"0"},"PeriodicalIF":4.1000,"publicationDate":"2023-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Content takedowns and activist organizing: Impact of social media content moderation on activists and organizing\",\"authors\":\"Diane Jackson\",\"doi\":\"10.1002/poi3.372\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Social media companies are increasingly transcending the offline sphere by shaping online discourse that has direct effects on offline outcomes. Recent polls have shown that as many as 70% of young people in the United States have used social media for information about political elections (Booth et al., 2020) and almost 30% of US adults have used social media to post about political and social issues (McClain, 2021). Further, social media have become a site of organizing with over half of US adults reporting having used social media as a tool for gathering or sharing information about political or social issues (Anderson et al., 2018). Despite the necessity of removing content that may breach the content guidelines set forth by social media companies such as Facebook, Instagram, Twitter, and TikTok, a gap persists between the content that violates guidelines and the content that is removed from social media sites. For activists particularly, content suppression is not only a matter of censorship at the individual level. During a time of significant mobilization, activists rely on their social media platforms perhaps more than ever before. This has been demonstrated by the Facebook Arabic page, “We Are All Khaled Said,” which has been credited with promoting the 2011 Egyptian Revolution (Alaimo, 2015). Activists posting about the Mahsa Amini protests and Ukrainians posting about the Russian invasion of Ukraine have reported similar experiences with recent Meta policy changes that have led to mass takedowns of protest footage and related content (Alimardani, 2022). The impacts of social media platforms' policy and practices for moderation are growing as cyberactivism has become more integral in social organizing (Cammaerts, 2015). However, due to accuracy and bias issues of content moderation algorithms deployed on these platforms (Binns et al., 2017; Buolamwini & Gebru, 2018; Rauchberg, 2022), engaging social media as a tool for social and political organizing is becoming more challenging. The intricacies and the downstream systemic effects of these content moderation techniques are not explicitly accounted for by social media platforms. Therefore, content moderation is pertinent to social media users based on the effects that moderation guidelines have not only on online behavior but also on offline behavior. The objectives of this paper are twofold. First and foremost, the goal of this paper is to contribute to the academic discourse raising awareness about how individuals are being silenced by content moderation algorithms on social media. This paper does this primarily by exploring the social and political implications of social media content moderation by framing them through the lens of activism and activist efforts online and offline. The secondary goal of this paper is to make a case for social media companies to develop features for individuals who are wrongfully marginalized on their platforms to be notified about and to appeal incidences of censorship and content removal. To do so, this paper will begin by discussing common techniques of content moderation including algorithmic content moderation, which various social media platforms have used to moderate content on their sites (Morrow et al., 2022). This paper will then address the impacts of algorithmic content moderation on its users generally, on individuals identifying with marginalized communities, and on activists by reviewing recent literature in this area and by citing recent examples of social movements impacted by social media censorship. Finally, this paper will consider the downstream systemic implications of algorithmic content moderation both politically and ethically for both social media users and members of society at large. Social media have evolved tremendously since their inception. With it, content moderation practices have also evolved (Morrow et al., 2022). However, social media companies, users, and regulators have different and sometimes competing motivations when it comes to content moderation (Morrow et al., 2022). As such, these practices have been criticized as inconsistent and lacking transparency (Ananny & Gillespie, 2016; Morrow et al., 2022) as they tend to produce significant errors that directly impact users (West, 2018). Further, these practices are seemingly not growing at a rate necessary to effectively and accurately accomplish the tasks for which it was developed while simultaneously expanding to reach the scale at which social media platforms are expanding. Although social media companies have not consistently or transparently moderated the content on their platforms or disclosed their moderation techniques (Bone, 2021; Morrow et al., 2022), mainstream social media platforms have tended to integrate an algorithmic content moderation technique derived from a manual coding process (Morrow et al., 2022). One such moderation approach involves moderating specific tags and keywords that may be damaging or inappropriate and is referred to as content labeling (see Gerrard, 2018). However, this approach has its drawbacks that make it impossible to function on its own. One recent study looked at proeating disorder content on image-sharing sites and online communities and found that users found ways of circumventing the hashtag and other forms of policing on the sites, emphasizing the limitations of the approach (see Gerrard, 2018). Though evidence of this still exists, it cannot sufficiently function alone. Other approaches incorporate more machine learning techniques and advanced pattern matching that compares new content to existing examples of training data developed by human coders and branded under the broad umbrella of artificial intelligence (AI; see Gillespie, 2020; Gorwa et al., 2020; Langvardt, 2018). This style of approach involves developers grouping together posts or comments as a sample of training data, which human coders manually code (Gillespie, 2020). At this point, the human coders feed the training data with its codes into a program, which uses machine learning techniques to extrapolate from the patterns of the coded training data to match other content in the social media platform (Gillespie, 2020). This format can be applied to both written content in social media posts, comments, and captions as well as to images or videos. The outcome of these approaches, however, seems to involve only presenting selected information and inappropriately omitting or hiding other information (Marcondes et al., 2021). Further, because this approach also incorporates features for users to flag or report offensive or inappropriate content, humans are still involved in the flagging of content by users and, sometimes, in the decision to remove the content (Gillespie, 2020). This paper will center around algorithmic content moderation generally and will weigh the benefits and drawbacks of these approaches before considering how they directly impact marginalized individuals and activist efforts. Algorithmic content moderation has two clear benefits: its reach and cost-effectiveness relative to compensating numerous human moderators who would be tasked with pouring over and flagging every offensive or inappropriate piece of content. The ever-growing size of data and simplistic judgments required to moderate content tend to be justifications that mainstream social media platforms use for the implementation of AI for content moderation (Gillespie, 2020; Gorwa et al., 2020). According to these companies, only AI can address the problems posed by the data, judgments, and user and content amount (Gillespie, 2020). While AI may address the issue of size (Gorwa et al., 2020) and may seem like an economical solution to protect human moderators from consuming damaging, hateful content and to preserve their time from making judgments that could be made by a computer (Gillespie, 2018, 2020), pervasive problems exist on these platforms that seem to mirror sociopolitical problems offline (Gorwa et al., 2020). Although algorithmic content moderation has been framed as the solution to many of the moderation issues that mainstream social media companies currently face and as an answer to legislative calls to address security and safety concerns on these platforms (Gorwa et al., 2020), this approach may present more entrenched problems than it addresses. Many of the existing content moderation algorithms discussed in recent literature seem to only engage AI in a broader sense, where much of the technology follows advanced methods of pattern matching of content first identified by human manual coders (Gorwa et al., 2020). In this sense, this technology is still dependent on humans to make initial judgments from which these tools follow inductive logic to flag similar or duplicate content instead of making original judgments itself. As such, the capability of these tools to identify inappropriate content stagnates rather than evolves over time with the trends of human language and interaction. Further, machine learning tools necessitate training data as indicators of classifying particular instances into specific categories (Duarte et al., 2017). The issue with the architecture of these tools is that these training data are necessary for the algorithms to identify similar or the same instances. However, the algorithms are fixed, then, to the similar or same instances of the training cases provided (Sinnreich, 2018). As such, the capacity for machine learning tools to account for cultural context, humor, subtlety, and other nuances that human judgment can identify is limited and fixed to the diversity of the messages captured by the training data it was provided (Duarte et al., 2017). This is particularly problematic due to the lack of representativeness that training data may have, the issues that these models tend to have with performing consistently across different domains, and the unrealistically rigid definitions that these tools apply when processing language (Duarte et al., 2017; Sinnreich, 2018). Lastly, because algorithmic content moderation is designed to remove content or hide it altogether, the lack of accuracy to correctly classify content is particularly damaging (Duarte et al., 2017), both for those who are sharing content that does not adhere with platform guidelines (Gerrard, 2018) and for those who are sharing guideline-adherent content (Marcondes et al., 2021; West, 2018). This is partly due to the lack of feasibility and ease with which users can circumvent platform moderation techniques. This was demonstrated by a case study documented by Gerrard (2018) of proeating disorder content shared by individuals whose content was not identified as damaging or harmful by the moderation algorithms of several mainstream photo-sharing and community-hosting social media sites. However, those whose content is being wrongly restricted or removed altogether are paying a price (Alimardani, 2022; Marcondes et al., 2021; Middlebrook, 2020; Morrow et al., 2022; West, 2018). As has been discussed in this section, these algorithmic moderation practices are only as accurate and unbiased as the individuals who have developed these practices and as up-to-date as the most recent update to these practices (Binns et al., 2017; Gorwa et al., 2020). As such, these practices are left open to the biased and systematic over-policing of certain groups and the neglect of others (Marcondes et al., 2021; Morrow et al., 2022; Sinnreich, 2018), which thus continually reinforces offline hegemonic principles (Middlebrook, 2020). This topic will be covered more in the following section. Machine learning tools rely on identifying and extracting patterns of behavior and responses from instances among groups of text and users. In doing so, the structure of machine learning necessitates the normalization of hegemonic principles and practices, which thus, can lead to the inaccurate and ineffective identification of these patterns among marginalized communities. As discussed by Gillespie (2020, p. 3), “the margin of error typically lands on the marginal: who these tools over-identify, or fail to protect, is rarely random.” Marginalization is defined in this paper as “both a condition and a process that prevents individuals and groups from full participation in social, economic, and political life enjoyed by the wider society” (Alakhunova et al., 2015, p. 10). Multiple recent analyses have been published about social media censorship and taking special note of the censorship of individuals from marginalized communities (see Alimardani, 2022; Dinar, 2021; Duarte et al., 2017; Knight Foundation & Gallup Inc., 2020; Smith et al., 2021). Specifically, analyses like these identify the key challenges, takeaways, and implications of the systemic censorship of individuals belonging to marginalized communities. One form of censorship identified broadly by these papers and other recent research articles is that of shadowbanning, or the algorithmic censorship, hiding, or removal of users' content who are said to have violated community guidelines and which results in many marginalized users' public content being hidden from feeds and searches without their awareness (Knight Foundation & Gallup Inc., 2020; Rauchberg, 2022; Smith et al., 2021). Twitter (Twitter Help Center, 2023) and Instagram (Instagram Help Center, 2023) both publicize on their sites the practice of making content less visible on more public locations such as hashtags or Explore pages or when searched. Due to the lack of a notification process for those whose content may be censored in this way, individuals are not given the opportunity to be made aware of their content or profiles being censored or how long the censorship will last. In a recent survey, almost every marginalized group reported experiencing censorship on Instagram at disproportionally higher percentages than more privileged groups (Smith et al., 2021). Specifically, this survey reported that sex workers, sex educators, LGBQIA+, trans, nonbinary, black, indigenous, and people of color, and disabled individuals all reported experiencing content removals from Instagram at higher rates than those in more privileged groups. Leaked internal documents from TikTok “instructed moderators to suppress posts created by users deemed too ugly, poor, or disabled for the platform” (Biddle et al., 2020, para. 1) and to censor political discourse during livestreams to attract new users. Further, TikTok has admitted to bias in its antibullying moderation system, which suppressed the content of individuals who appeared to have a disability, facial disfigurement, or other characteristics that may make them vulnerable to being bullied (Botella, 2019; Köver & Reuter, 2019; Rauchberg, 2022). As such, the app's brief history has been riddled with evidence of the algorithmic suppression of certain groups. Other leaked information about the company revealed the tactics that TikTok's parent company, ByteDance, has used to moderate, suppress, or censor content that goes against China's foreign policy goals (Hern, 2019). This evidence shows a moderation of content about Tiananmen Square and Tibetan independence (Hern, 2019) and substantiates accusations against the company for censoring content about the Hong Kong protests (Harwell & Romm, 2019). Another research article about shadowbanning marginalized individuals underscores how more vulnerable communities are being disproportionately targeted by these moderation techniques on- and offline via Instagram (Middlebrook, 2020). This piece considered the implications that marginalized communities face being shadowbanned given the lack of awareness that users generally have about moderation algorithms. Unfortunately, the lack of guidelines provided to users to know when or why they are being shadowbanned and the lack of an appeals process to reverse the shadowban make this a particularly unjust penalization. Censorship and surveillance adversely affect users in marginalized communities and activists (Asher-Schapiro, 2022) who rely on social media to organize (Alimardani, 2022; Sandoval-Almazan & Ramon Gil-Garcia, 2014). For activists and individuals in marginalized communities, this online oppression can be the difference-maker for their organizing efforts and information sharing to be successful. The lack of specificity that social media companies offer around these practices and the lack of accuracy that social media content moderation algorithms can have when moderating the content of marginalized individuals has major implications on users' freedom of speech. The lack of transparency into moderation algorithms and recent changes to social media sites (e.g., Elon Musk purchasing Twitter; Asher-Schapiro, 2022) have prompted more discourse regarding the way social media sites moderate conversations (see Gerrard, 2018) and who are actually being moderated (Middlebrook, 2020). These systemic inadequacies of moderation algorithms are exacerbated by the common practices of most sites like these to depend on their users to report content as offensive or inappropriate instead of detecting the content through other methods (Chen et al., 2012). Relying on and entrusting users to report content becomes a greater issue of suppressing marginalized, guideline-adherent content. Users could exploit their privileges to report content for removal regardless of whether the content adheres to the site's community guidelines. Existing research about partisan-motivated behaviors like distributing fake news articles (Osmundsen et al., 2021) and labeling news as fake when it disconfirms partisans' existing beliefs (Kahne & Bowyer, 2017) offers support for the possibility that social media users may use features to report content for removal when it contradicts their attitudes. As such, activists may be up against a double bind where the automated content moderation systems in place may unfairly and inaccurately remove their content automatically and where individuals who disagree with them may flag their content to these inequitable automated systems for removal. This double bind puts activists who use social media at a disadvantage in their efforts to share their messages, despite social media becoming increasingly more necessary to engage in activism (Cammaerts, 2015). This section will discuss two recent and important examples of social movements that used social media in unique and important ways and the effect that content takedowns had or could have had on the movements. The BLM movement originated with the social media hashtag #BlackLivesMatter after George Zimmerman, the man who shot and killed Trayvon Martin in 2012, was acquitted. The hashtag and movement gained tremendous momentum in the United States in 2014 after Michael Brown in Missouri and Eric Garner in New York were killed in incidents of police brutality (Howard University School of Law, 2023). The hashtag and movement spread globally after the police murder of George Floyd in Minnesota in 2020 (Howard University School of Law, 2023; Tilly et al., 2019). In a study about collegiate athlete activism in the United States, collegiate athlete activists considered social media as one of the primary modalities through which activism occurs and through which activists can reach, amplify, and engage with messages from other members of their movements (Feder et al., 2023). For some, demonstrations took place exclusively on social media and for others, social media was an integral component of amplifying the calls to action that they publicized. These activists' organizing efforts manifested in players' strikes, donation drives after riots in the wake of the murder of George Floyd, voting campaigns, and even sales of BLM masks, the proceeds from which were donated to charities geared toward helping marginalized communities (Feder et al., 2023). Censoring the messages of these activists during their organizing efforts and during attempts to mobilize becomes an issue of political freedom as well as free speech. These experiences of suppressing political freedom and free speech were voiced by BLM activists who used TikTok during the height of BLM demonstrations that came about after the murder of George Floyd (McCluskey, 2020). These activists described drastic reductions of appearances that their videos had on users' “For You” pages on TikTok (McCluskey, 2020; Nicholas, 2022). After becoming a central site for sharing activism content (Janfaza, 2020), TikTok released a statement apologizing to its community of Black creators and users who had previously voiced feelings of marginalization and suppression on the app (see Pappas & Chikumbu, 2020). According to the official statement released by TikTok, Black content creators and allies who are users of the app changed their profile pictures and spoke out through the platform to discuss their experiences of marginalization on TikTok (Pappas & Chikumbu, 2020). The statement went on to explain that “a technical glitch made it temporarily appear as if posts uploaded using #BlackLivesMatter and #GeorgeFloyd would receive 0 views” (para. 3). Unfortunately, almost 2 months after the release of this statement, Black TikTok creators voiced continued experiences of content suppression and marginalization on the app (McCluskey, 2020). BLM activists who use the app reported not only lacking visibility to other #BlackLivesMatter content on their personalized feeds, but racist and offensive comments shared by individuals who were shown their BLM content (Allyn, 2020). Black creators who returned to posting non-BLM content on TikTok found that their regular content was receiving far less engagement (McCluskey, 2020). Further, one Black creator, Emily Barbour, recounted to TIME Magazine having created a post from a screen recording that she took containing another TikTok video in which the creator appeared in blackface in an effort to call out the video (McCluskey, 2020). While Barbour's video was hidden for being deemed a copyright violation, TikTok did not consider the video featuring a person in blackface as violating its community guidelines, leaving the video up for two more days despite thousands of users having flagged the video (McCluskey, 2020). Barbour shared with TIME the experience of racial bias and unfair treatment on TikTok (McCluskey, 2020). TikTok is not the only social media platform with documented instances of suppressing content from BLM activists. Louiza Doran is a antiracism activist and educator who experienced censorship on Instagram (Silverman, 2020). According to Doran in an interview with BuzzFeed News, her account was prohibited from livestreaming and some of her posts and comments were removed altogether (Silverman, 2020). Having been described by a spokesperson from Facebook as a technical issue that resulted in Doran's Facebook and Instagram accounts being flagged as spam (Silverman, 2020), the explanation of this issue bears striking resemblance to TikTok's “technical glitch” that caused BLM-related posts to be displayed differently (Pappas & Chikumbu, 2020). This common thread of social media companies blaming technical bugs as reasons for suppressing and censoring content points to: (1) the shortcomings of algorithmic content moderation, (2) the lack of visibility that social media companies have into the downstream effects of these moderation systems, and (3) the scapegoating strategy that these platforms use to dismiss concerns of systemic moderation bias as technological difficulty. For activists and marginalized individuals, these are not just setbacks but critical obstacles that encumber not only these people, but their followers, and, thus, factions of entire movements. The Mahsa Amini protests in Iran, which have sparked the largest antiregime uprising the country has seen since the Iranian Revolution in 1979 (France-Presse, 2022), have relied on features like hashtags and encryption on platforms like Instagram and Twitter to spread awareness and amplify messages of the protests within and outside of Iran (Amidi, 2022; Kenyon, 2023; Kumar, 2022). These antiregime protests began in 2022 in Iran and have continued as of September 2023 (Kenyon, 2023), and messages of this movement have spread across the world. Activists have been forced to rapidly transition between social media sites to counteract the Iranian government's efforts to suppress the movement's messages (Amidi, 2022; Iran International, 2022; Kumar, 2022). Due to the Iranian government's response to suppress the movement and its actors with violence, censorship of social media posts, and internet outages (CNN Special Report, 2023; France-Presse, 2022), Iranian protesters have had to depend on individuals outside of the country as well as innovations like encryption and offsite servers to share protest information on social media and to stay up-to-date (Amidi, 2022; Butcher, 2022; Kumar, 2022). Amidst the governmental censorship and violence against protestors (Amidi, 2022; CNN Special Report, 2023), the movement and its members have demonstrated resilience by quickly pivoting between social media platforms and by amplifying its messages through their online and offline presence (Kumar, 2022). As such, social media has played a critical role in the movement's sustained impact (Amidi, 2022; Kumar, 2022). Specifically, Instagram has become one of Iran's most popular social media platforms and it is the only uncensored foreign form of social media (Alimardani, 2022; Dagres, 2021). However, a policy change that Meta made in 2022 in response to the Russian invasion of Ukraine has led to the removal of large amounts of Iranian protest-related content that contained a common Iranian protest slogan (Alimardani, 2022). This slogan, which translates into English as calls for death to the military forces, current supreme leader, and current president, exist culturally as a symbolic dissident call against Iran's authoritarian regime rather than a true call for violence (Alimardani, 2022). Where Meta used to include these slogans in their exceptions of community guidelines, a new policy change that Meta made rolls back these exceptions both for these slogans in Iran and for other protest slogans used in Ukraine during the Russian invasion (Alimardani, 2022; Biddle, 2022; Vengattil & Culliford, 2022). As such, Meta has been criticized for prioritizing more respect for the dictators than for the protestors (Höppner, 2022) where their policies regarding human rights and free speech more align with U.S. policy (Biddle, 2022). For Iranian protesters, their government's capabilities paired with the policy decisions that social media companies like Meta make in moderating their content pose a unique combined challenge for organizing and mobilizing. This mass systematic removal of posts from personal accounts and important national media outlets that came about without warning in the wake of Meta's policy change (Alimardani, 2022) has exacerbated the extent to which these protesters are forced to monitor their posts and strategize their technology use. Further, evidence of the Iranian government hacking activists' accounts or taking control of the accounts of protesters in their custody (Polglase & Mezzofiore, 2022) pose significant threats to activists and to their movement. Indeed, social media platforms' moderation policies and practices have important implications on activists and their movements. This Meta content moderation policy change has produced a similar outcome to the censorship practiced by the Iranian government. As such, this situation speaks to the necessity for social media platforms to identify and institute culturally nuanced understanding in their moderation policies and practices if their objectives truly involve promoting free speech on their sites. In addition to the concerns that algorithmic moderation systems perpetuate systemic political injustices that persist offline, these practices come with numerous ethical concerns and issues of scale (Gorwa et al., 2020; Marcondes et al., 2021). Therefore, the following sections will review political and ethical concerns and recommendations specifically as these algorithmic moderation practices pertain to activism and organizing. There are many political implications of the current structure of algorithmic content moderation that should be considered. Because social media companies' content moderation algorithms' display of bias against marginalized individuals has been documented (Middlebrook, 2020; Rauchberg, 2022) and because activists are often individuals who identify with at least one marginalized group (Heaney, 2022; Kluch, 2022), erroneously removing content shared by individuals who are activists and who are marginalized exacerbates the marginalization (Middlebrook, 2020). Societally, this practice stunts movements, hampers representation, targets groups, silences individuals, violates rights to free speech, and multiplies the workload of the content creators who identify in these ways and experience this mistreatment. In addition to the specific issues that are considered in activists' posts, these individual implications carry weight for political engagement, political information seeking, and public policy. Assumptions of western ideals such as freedom of speech and freedom of the press underlie the discussion in this paper. As polarization across party lines on sociopolitical issues continues to grow, partisan behaviors such as sharing news articles that disparage opposing political parties and party members on social media have persisted (Osmundsen et al., 2021). Of course, reporting content is now feasibly emerging as another possible partisan behavior that works to disparage opposing perspectives by flagging them for removal. However, these behaviors are not only centralized around western countries where apparent government censorship may not be present because censorship may come from sources other than government entities (e.g., host social media companies or third parties). For activists in countries where government censorship of social media content is exerted (see Casilli & Tubaro, 2012; Golovchenko, 2022; Liu & Zhao, 2021; Poell, 2014; Tai & Fu, 2020), activists' labor is multiplied to both create and publicize their messages, mobilize individuals, and replicate this process to combat either the algorithmic content removal or governmental censorship. Because the individuals who create and deploy algorithms are embedded in systems of oppression, their products reproduce these systems (Middlebrook, 2020), which ultimately oppresses activists and marginalized individuals, thereby oppressing social movements and organizing processes, political discourse, political engagement, and, thus, political outcomes. As such, even activists who reside in countries that tout the prioritization of free speech are experiencing instances of censorship (Langvardt, 2018). Instances like these where certain perspectives are excluded from discourse have been shown to impact others' perceptions of public opinion and marginalize the expression of those with similar positions (Gearhart & Zhang, 2015; Roy, 2017). Ethical concerns for algorithmic content moderation practices involve democratizing content removal features, the relational and emotional ramifications of these systems, and the lack of cultural and linguistic understanding that these practices demonstrate. Enabling all users to report content for removal broadens the potential for innocent content to be removed automatically by these moderation algorithms as it puts the power in users' hands, thereby increasing the amount of flagged content that is appropriate and, thus, the number of ambiguous cases that human moderators would potentially need to review. Though social media companies have focused on effectively reducing manual moderation while keeping hate speech and cyberbullying at bay, the overreach of these algorithms relative to the inaccuracy of these moderation algorithms has not been considered (Center for Countering Digital Hate, 2022). However, existing research has also documented relational and emotional ramifications that individuals who report having been censored and shadowbanned experience online and offline (West, 2018). What many may not also recognize is the emotional toll that is placed on both human moderators who work for social media companies (Gillespie, 2018, 2020) and those who consume content that should be removed and was not detected by algorithms (Center for Countering Digital Hate, 2022). It is critical to recognize the emotional labor that human content moderators tasked with categorizing content into appropriate content and inappropriate content in the building of training datasets are confronted with (Gillespie, 2018; Sinnreich, 2018). Further, although human content moderators can identify ironic statements and greater nuance than moderation algorithms (Duarte et al., 2017), human content moderation applies the same level of bias to what is considered hate speech or inappropriate content with potentially more inconsistency. Given the socially constructed nature of hate speech and what it is or is not, the notion of hate speech may differ culturally and cannot be fixed to a singular point in time (Gillespie, 2020). Meta's content moderation policy change that led to the mass removal of posts using Iran's dissident slogan is an important example of this (Alimardani, 2022). Meta has also demonstrated an incapacity for moderating content in non-Western languages and applying Western ideals to moderation policies in other parts of the world (Alimardani & Elswah, 2021; Fatafta, 2022). Thus, Meta has demonstrated a lack of cultural sensitivity in its creation and institution of moderation policy and practices. These issues, however, may be ameliorated by using moderation algorithms to flag the most egregious and universally held forms of offensive speech and promoting more ambiguous posts to human coders (Gillespie, 2020). This change and increased transparency of social media companies about the moderation algorithms they deploy and content removal for users are meaningful developments that social media platforms could offer to help wrongfully censored content be corrected. Although each of these implications are significant, issues such as the lack of transparency into the innerworkings of social media moderation algorithms and the lack of structural opportunities to appeal or inquire about moderation practices are some of the most integral. Despite the fact that experts in AI have developed ethics guidelines for users and for developers (see Ryan & Stahl [2020] for one example), social media platforms such as Instagram or Twitter do not seem to publicize or practice ethical competencies that guide their algorithm development and implementation. In fact, with Elon Musk's purchase of Twitter, the team dedicated to ethically practicing AI and promoting transparency and fairness was reportedly disbanded (Knight, 2022). Social media companies should also broaden their organizational structure so that developers institute moderation practices specific to the regions and languages of which they demonstrate sufficient cultural understanding. By offering more transparent affordances such as notifications and appeals features for users whose content is removed or censored from searches, users can be made aware of how their content is violating regulations and appeal these decisions. Because Twitter, Instagram, and Facebook have notifications and appeals options in place for other instances, these companies already have the infrastructure that could be extended to allow for these possibilities. Further, algorithms could be updated as coders review appeals in an effort to protect and better represent the groups of people like activists and marginalized individuals whose content tends to be removed more often and unjustly. Additional features could involve implementing a business profile category for users to label their accounts as political activists. Social media companies could then vet the profiles and institute exemptions in their content moderation algorithms specifically for accounts with this profile category type. The first goal of this paper was to contribute to the academic discourse sharing more information about how certain individuals are being censored and silenced by social content moderation algorithms. To do this, this paper considered the various forms of social media content moderation along with how these features work in tension with free speech and equal treatment of user groups that are espoused by these platforms. The second goal of this paper was to propose for social media companies to implement greater transparency into their content moderation practices. This paper recommends extending their current appeals infrastructure to notify users when their content or accounts are being hidden from searches or public pages, changing the structure of algorithmic content moderation, and offering additional features to make moderation more accurate. By implementing algorithms for more extreme, offensive content, enlisting human moderators to code more ambiguous, nuanced posts (Gillespie, 2020) and building algorithms to be more iterative so that they can be updated as coders review appeals to automated moderation decisions, moderation algorithms will have a much greater capacity for ethically, accurately, and effectively moderating hate speech while promoting free speech equally. Social media companies have a way to go before arriving at a finish line for ethically, accurately, and effectively mitigating hate speech and encouraging political and social discourse. Major recurring issues such as the lack of transparency of the innerworkings of social media algorithmic moderation systems and the lack of structural opportunities to appeal or inquire about moderation practices are only the tip of the iceberg. As marginalized communities are targeted through automated moderation practices like shadowbanning, oppressive and inequitable treatment of these users is reinforced online while offline hegemonic systems that silence marginalized communities grow stronger. The issue of social media content moderation—particularly algorithmic content takedowns—and how they affect social activists who are using social media as a tool for organizing is of significance both for social justice movements and their members as well as social media platforms and their stakeholders. As cyberactivism and the online amplification of offline social issues integrate with physical mobilizing (Sandoval-Almazan & Ramon Gil-Garcia, 2014), the role of social media in supporting social activists' online communication is increasingly important. Equipping all users, regardless of the extreme ideological positions that they may carry or vocalize, with these tools further exacerbates the gap between offensive or inappropriate content and removed content. Doing so runs counter to the notion of free speech, thereby making this capability an issue of public policy. Indeed, making features like this widely accessible does more to reinforce the offline hegemony online and contributes more to engaging in free content flagging than it does to engaging in free speech.\",\"PeriodicalId\":46894,\"journal\":{\"name\":\"Policy and Internet\",\"volume\":\"3 3\",\"pages\":\"0\"},\"PeriodicalIF\":4.1000,\"publicationDate\":\"2023-10-26\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Policy and Internet\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1002/poi3.372\",\"RegionNum\":1,\"RegionCategory\":\"文学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMMUNICATION\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Policy and Internet","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1002/poi3.372","RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMMUNICATION","Score":null,"Total":0}
引用次数: 0

摘要

此外,TikTok承认其反欺凌审核系统存在偏见,该系统压制了那些似乎有残疾、面部毁容或其他可能使他们容易受到欺凌的特征的个人的内容(博特拉,2019;Köver &路透社,2019;Rauchberg, 2022)。因此,这款应用的短暂历史中充斥着算法压制某些群体的证据。其他泄露的有关该公司的信息揭示了抖音母公司字节跳动(ByteDance)用来缓和、压制或审查违背中国外交政策目标的内容的策略(Hern, 2019)。这一证据表明,有关天安门广场和西藏独立的内容有所节制(Hern, 2019),并证实了对该公司审查有关香港抗议活动内容的指控(Harwell & Romm, 2019)。另一篇关于影子禁止边缘化个人的研究文章强调了更脆弱的社区是如何通过Instagram在线上和离线受到这些节制技术的不成比例的攻击的(Middlebrook, 2020)。这篇文章考虑了由于用户通常缺乏对审核算法的认识,边缘化社区面临被影子封禁的影响。不幸的是,缺乏指导方针让用户知道他们何时或为什么被禁止使用影子,也缺乏撤销影子禁令的上诉程序,这使得这种惩罚特别不公正。审查和监视对依赖社交媒体组织的边缘化社区和活动家的用户和活动家(Asher-Schapiro, 2022)产生不利影响(Alimardani, 2022;Sandoval-Almazan & Ramon Gil-Garcia, 2014)。对于边缘化社区的活动人士和个人来说,这种网络压迫可以成为他们组织努力和信息共享成功的关键因素。社交媒体公司对这些做法缺乏专一性,社交媒体内容审核算法在审核边缘化个人的内容时缺乏准确性,这对用户的言论自由产生了重大影响。审核算法缺乏透明度,以及最近社交媒体网站的变化(例如,埃隆·马斯克(Elon Musk)收购Twitter;Asher-Schapiro, 2022)引发了更多关于社交媒体网站如何调节对话(见Gerrard, 2018)以及谁实际上正在被调节(Middlebrook, 2020)的讨论。这些审核算法的系统性不足被大多数网站的常见做法所加剧,这些网站依赖用户报告内容为冒犯性或不适当,而不是通过其他方法检测内容(Chen et al., 2012)。依赖和委托用户报告内容成为压制边缘化、遵循指导方针的内容的更大问题。用户可以利用他们的特权来报告内容,无论内容是否遵守网站的社区准则。关于党派动机行为的现有研究,如分发假新闻文章(Osmundsen等人,2021),以及当新闻不符合党派成员的现有信念时将其标记为假新闻(Kahne & Bowyer, 2017),为社交媒体用户可能会使用功能来报告内容以删除与他们的态度相矛盾的内容的可能性提供了支持。因此,活动人士可能会面临双重困境,即现有的自动内容审核系统可能不公平地、不准确地自动删除他们的内容,而不同意他们的个人可能会向这些不公平的自动系统标记他们的内容,以便删除。这种双重束缚使得使用社交媒体的活动家在分享他们的信息时处于不利地位,尽管社交媒体对参与激进主义越来越必要(Cammaerts, 2015)。本节将讨论最近两个重要的社会运动的例子,这些运动以独特而重要的方式使用社交媒体,以及内容删除对运动产生或可能产生的影响。在2012年枪杀特雷沃恩·马丁的乔治·齐默尔曼被判无罪后,BLM运动起源于社交媒体上的标签#黑人的生命也很重要#。2014年,密苏里州的迈克尔·布朗和纽约的埃里克·加纳在警察暴力事件中丧生后,这一标签和运动在美国获得了巨大的动力(霍华德大学法学院,2023)。在2020年明尼苏达州警察谋杀乔治·弗洛伊德(George Floyd)之后,这个标签和运动在全球蔓延开来(霍华德大学法学院,2023年;Tilly等人,2019)。在一项关于美国大学运动员行动主义的研究中,大学运动员活动家认为社交媒体是行动主义发生的主要方式之一,通过社交媒体,活动家可以接触、放大和参与来自其他运动成员的信息(Feder et al., 2023)。 活动人士被迫在社交媒体网站之间迅速转换,以对抗伊朗政府压制该运动信息的努力(Amidi, 2022;伊朗国际,2022年;库马尔,2022)。由于伊朗政府以暴力镇压运动及其参与者,审查社交媒体帖子和互联网中断(CNN特别报道,2023;法新社,2022),伊朗抗议者不得不依靠国外的个人以及加密和异地服务器等创新,在社交媒体上分享抗议信息,并保持最新状态(Amidi, 2022;屠夫,2022;库马尔,2022)。在政府审查和对抗议者的暴力中(Amidi, 2022;CNN特别报道,2023),该运动及其成员通过在社交媒体平台之间快速切换,并通过在线和离线存在放大其信息,展示了弹性(Kumar, 2022)。因此,社交媒体在运动的持续影响中发挥了关键作用(Amidi, 2022;库马尔,2022)。具体来说,Instagram已经成为伊朗最受欢迎的社交媒体平台之一,它是唯一未经审查的外国社交媒体形式(Alimardani, 2022;达格瑞斯,2021)。然而,为了应对俄罗斯入侵乌克兰,Meta在2022年做出了一项政策调整,导致大量与伊朗抗议相关的内容被删除,这些内容包含了一个常见的伊朗抗议口号(Alimardani, 2022)。这个口号,翻译成英语是呼吁军队、现任最高领导人和现任总统去死,在文化上作为反对伊朗独裁政权的象征性异议呼吁而存在,而不是真正的暴力呼吁(Alimardani, 2022)。Meta过去曾将这些口号纳入其社区指导方针的例外中,但Meta做出的一项新政策改变,使这些例外既适用于伊朗的这些口号,也适用于俄罗斯入侵期间在乌克兰使用的其他抗议口号(Alimardani, 2022;比德尔,2022;Vengattil & Culliford, 2022)。因此,Meta被批评优先尊重独裁者而不是抗议者(Höppner, 2022),而他们在人权和言论自由方面的政策更符合美国的政策(Biddle, 2022)。对伊朗抗议者来说,政府的能力加上Meta等社交媒体公司在审核他们的内容时做出的政策决定,对组织和动员构成了独特的综合挑战。在Meta的政策改变之后,这种大规模系统性地从个人账户和重要的国家媒体上删除帖子的行为毫无预警(Alimardani, 2022),加剧了这些抗议者被迫监控自己的帖子和制定技术使用策略的程度。此外,伊朗政府黑客攻击活动人士账户或控制被其拘留的抗议者账户的证据(Polglase & Mezzofiore, 2022)对活动人士及其运动构成了重大威胁。事实上,社交媒体平台的节制政策和做法对活动人士及其运动有着重要的影响。这项Meta内容审核政策的改变,产生了与伊朗政府实行的审查制度类似的结果。因此,这种情况说明,如果社交媒体平台的目标真的涉及在其网站上促进言论自由,就有必要在其审核政策和实践中识别并建立文化上细微差别的理解。除了担心算法节制系统会延续离线持续存在的系统性政治不公正之外,这些做法还带来了许多道德问题和规模问题(Gorwa et al., 2020;Marcondes et al., 2021)。因此,以下部分将审查政治和道德问题和建议,特别是这些算法节制实践与行动主义和组织有关。目前的算法内容审核结构有许多应该考虑的政治影响。因为社交媒体公司的内容审核算法对边缘化个体的偏见已经被记录在案(Middlebrook, 2020;Rauchberg, 2022),因为积极分子通常是至少认同一个边缘化群体的个人(Heaney, 2022;Kluch, 2022),错误地删除由积极分子和被边缘化的个人共享的内容加剧了边缘化(Middlebrook, 2020)。在社会上,这种做法阻碍了运动,阻碍了代表,针对群体,使个人沉默,侵犯了言论自由权,并增加了内容创作者的工作量,他们以这些方式识别并经历了这种虐待。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Content takedowns and activist organizing: Impact of social media content moderation on activists and organizing
Social media companies are increasingly transcending the offline sphere by shaping online discourse that has direct effects on offline outcomes. Recent polls have shown that as many as 70% of young people in the United States have used social media for information about political elections (Booth et al., 2020) and almost 30% of US adults have used social media to post about political and social issues (McClain, 2021). Further, social media have become a site of organizing with over half of US adults reporting having used social media as a tool for gathering or sharing information about political or social issues (Anderson et al., 2018). Despite the necessity of removing content that may breach the content guidelines set forth by social media companies such as Facebook, Instagram, Twitter, and TikTok, a gap persists between the content that violates guidelines and the content that is removed from social media sites. For activists particularly, content suppression is not only a matter of censorship at the individual level. During a time of significant mobilization, activists rely on their social media platforms perhaps more than ever before. This has been demonstrated by the Facebook Arabic page, “We Are All Khaled Said,” which has been credited with promoting the 2011 Egyptian Revolution (Alaimo, 2015). Activists posting about the Mahsa Amini protests and Ukrainians posting about the Russian invasion of Ukraine have reported similar experiences with recent Meta policy changes that have led to mass takedowns of protest footage and related content (Alimardani, 2022). The impacts of social media platforms' policy and practices for moderation are growing as cyberactivism has become more integral in social organizing (Cammaerts, 2015). However, due to accuracy and bias issues of content moderation algorithms deployed on these platforms (Binns et al., 2017; Buolamwini & Gebru, 2018; Rauchberg, 2022), engaging social media as a tool for social and political organizing is becoming more challenging. The intricacies and the downstream systemic effects of these content moderation techniques are not explicitly accounted for by social media platforms. Therefore, content moderation is pertinent to social media users based on the effects that moderation guidelines have not only on online behavior but also on offline behavior. The objectives of this paper are twofold. First and foremost, the goal of this paper is to contribute to the academic discourse raising awareness about how individuals are being silenced by content moderation algorithms on social media. This paper does this primarily by exploring the social and political implications of social media content moderation by framing them through the lens of activism and activist efforts online and offline. The secondary goal of this paper is to make a case for social media companies to develop features for individuals who are wrongfully marginalized on their platforms to be notified about and to appeal incidences of censorship and content removal. To do so, this paper will begin by discussing common techniques of content moderation including algorithmic content moderation, which various social media platforms have used to moderate content on their sites (Morrow et al., 2022). This paper will then address the impacts of algorithmic content moderation on its users generally, on individuals identifying with marginalized communities, and on activists by reviewing recent literature in this area and by citing recent examples of social movements impacted by social media censorship. Finally, this paper will consider the downstream systemic implications of algorithmic content moderation both politically and ethically for both social media users and members of society at large. Social media have evolved tremendously since their inception. With it, content moderation practices have also evolved (Morrow et al., 2022). However, social media companies, users, and regulators have different and sometimes competing motivations when it comes to content moderation (Morrow et al., 2022). As such, these practices have been criticized as inconsistent and lacking transparency (Ananny & Gillespie, 2016; Morrow et al., 2022) as they tend to produce significant errors that directly impact users (West, 2018). Further, these practices are seemingly not growing at a rate necessary to effectively and accurately accomplish the tasks for which it was developed while simultaneously expanding to reach the scale at which social media platforms are expanding. Although social media companies have not consistently or transparently moderated the content on their platforms or disclosed their moderation techniques (Bone, 2021; Morrow et al., 2022), mainstream social media platforms have tended to integrate an algorithmic content moderation technique derived from a manual coding process (Morrow et al., 2022). One such moderation approach involves moderating specific tags and keywords that may be damaging or inappropriate and is referred to as content labeling (see Gerrard, 2018). However, this approach has its drawbacks that make it impossible to function on its own. One recent study looked at proeating disorder content on image-sharing sites and online communities and found that users found ways of circumventing the hashtag and other forms of policing on the sites, emphasizing the limitations of the approach (see Gerrard, 2018). Though evidence of this still exists, it cannot sufficiently function alone. Other approaches incorporate more machine learning techniques and advanced pattern matching that compares new content to existing examples of training data developed by human coders and branded under the broad umbrella of artificial intelligence (AI; see Gillespie, 2020; Gorwa et al., 2020; Langvardt, 2018). This style of approach involves developers grouping together posts or comments as a sample of training data, which human coders manually code (Gillespie, 2020). At this point, the human coders feed the training data with its codes into a program, which uses machine learning techniques to extrapolate from the patterns of the coded training data to match other content in the social media platform (Gillespie, 2020). This format can be applied to both written content in social media posts, comments, and captions as well as to images or videos. The outcome of these approaches, however, seems to involve only presenting selected information and inappropriately omitting or hiding other information (Marcondes et al., 2021). Further, because this approach also incorporates features for users to flag or report offensive or inappropriate content, humans are still involved in the flagging of content by users and, sometimes, in the decision to remove the content (Gillespie, 2020). This paper will center around algorithmic content moderation generally and will weigh the benefits and drawbacks of these approaches before considering how they directly impact marginalized individuals and activist efforts. Algorithmic content moderation has two clear benefits: its reach and cost-effectiveness relative to compensating numerous human moderators who would be tasked with pouring over and flagging every offensive or inappropriate piece of content. The ever-growing size of data and simplistic judgments required to moderate content tend to be justifications that mainstream social media platforms use for the implementation of AI for content moderation (Gillespie, 2020; Gorwa et al., 2020). According to these companies, only AI can address the problems posed by the data, judgments, and user and content amount (Gillespie, 2020). While AI may address the issue of size (Gorwa et al., 2020) and may seem like an economical solution to protect human moderators from consuming damaging, hateful content and to preserve their time from making judgments that could be made by a computer (Gillespie, 2018, 2020), pervasive problems exist on these platforms that seem to mirror sociopolitical problems offline (Gorwa et al., 2020). Although algorithmic content moderation has been framed as the solution to many of the moderation issues that mainstream social media companies currently face and as an answer to legislative calls to address security and safety concerns on these platforms (Gorwa et al., 2020), this approach may present more entrenched problems than it addresses. Many of the existing content moderation algorithms discussed in recent literature seem to only engage AI in a broader sense, where much of the technology follows advanced methods of pattern matching of content first identified by human manual coders (Gorwa et al., 2020). In this sense, this technology is still dependent on humans to make initial judgments from which these tools follow inductive logic to flag similar or duplicate content instead of making original judgments itself. As such, the capability of these tools to identify inappropriate content stagnates rather than evolves over time with the trends of human language and interaction. Further, machine learning tools necessitate training data as indicators of classifying particular instances into specific categories (Duarte et al., 2017). The issue with the architecture of these tools is that these training data are necessary for the algorithms to identify similar or the same instances. However, the algorithms are fixed, then, to the similar or same instances of the training cases provided (Sinnreich, 2018). As such, the capacity for machine learning tools to account for cultural context, humor, subtlety, and other nuances that human judgment can identify is limited and fixed to the diversity of the messages captured by the training data it was provided (Duarte et al., 2017). This is particularly problematic due to the lack of representativeness that training data may have, the issues that these models tend to have with performing consistently across different domains, and the unrealistically rigid definitions that these tools apply when processing language (Duarte et al., 2017; Sinnreich, 2018). Lastly, because algorithmic content moderation is designed to remove content or hide it altogether, the lack of accuracy to correctly classify content is particularly damaging (Duarte et al., 2017), both for those who are sharing content that does not adhere with platform guidelines (Gerrard, 2018) and for those who are sharing guideline-adherent content (Marcondes et al., 2021; West, 2018). This is partly due to the lack of feasibility and ease with which users can circumvent platform moderation techniques. This was demonstrated by a case study documented by Gerrard (2018) of proeating disorder content shared by individuals whose content was not identified as damaging or harmful by the moderation algorithms of several mainstream photo-sharing and community-hosting social media sites. However, those whose content is being wrongly restricted or removed altogether are paying a price (Alimardani, 2022; Marcondes et al., 2021; Middlebrook, 2020; Morrow et al., 2022; West, 2018). As has been discussed in this section, these algorithmic moderation practices are only as accurate and unbiased as the individuals who have developed these practices and as up-to-date as the most recent update to these practices (Binns et al., 2017; Gorwa et al., 2020). As such, these practices are left open to the biased and systematic over-policing of certain groups and the neglect of others (Marcondes et al., 2021; Morrow et al., 2022; Sinnreich, 2018), which thus continually reinforces offline hegemonic principles (Middlebrook, 2020). This topic will be covered more in the following section. Machine learning tools rely on identifying and extracting patterns of behavior and responses from instances among groups of text and users. In doing so, the structure of machine learning necessitates the normalization of hegemonic principles and practices, which thus, can lead to the inaccurate and ineffective identification of these patterns among marginalized communities. As discussed by Gillespie (2020, p. 3), “the margin of error typically lands on the marginal: who these tools over-identify, or fail to protect, is rarely random.” Marginalization is defined in this paper as “both a condition and a process that prevents individuals and groups from full participation in social, economic, and political life enjoyed by the wider society” (Alakhunova et al., 2015, p. 10). Multiple recent analyses have been published about social media censorship and taking special note of the censorship of individuals from marginalized communities (see Alimardani, 2022; Dinar, 2021; Duarte et al., 2017; Knight Foundation & Gallup Inc., 2020; Smith et al., 2021). Specifically, analyses like these identify the key challenges, takeaways, and implications of the systemic censorship of individuals belonging to marginalized communities. One form of censorship identified broadly by these papers and other recent research articles is that of shadowbanning, or the algorithmic censorship, hiding, or removal of users' content who are said to have violated community guidelines and which results in many marginalized users' public content being hidden from feeds and searches without their awareness (Knight Foundation & Gallup Inc., 2020; Rauchberg, 2022; Smith et al., 2021). Twitter (Twitter Help Center, 2023) and Instagram (Instagram Help Center, 2023) both publicize on their sites the practice of making content less visible on more public locations such as hashtags or Explore pages or when searched. Due to the lack of a notification process for those whose content may be censored in this way, individuals are not given the opportunity to be made aware of their content or profiles being censored or how long the censorship will last. In a recent survey, almost every marginalized group reported experiencing censorship on Instagram at disproportionally higher percentages than more privileged groups (Smith et al., 2021). Specifically, this survey reported that sex workers, sex educators, LGBQIA+, trans, nonbinary, black, indigenous, and people of color, and disabled individuals all reported experiencing content removals from Instagram at higher rates than those in more privileged groups. Leaked internal documents from TikTok “instructed moderators to suppress posts created by users deemed too ugly, poor, or disabled for the platform” (Biddle et al., 2020, para. 1) and to censor political discourse during livestreams to attract new users. Further, TikTok has admitted to bias in its antibullying moderation system, which suppressed the content of individuals who appeared to have a disability, facial disfigurement, or other characteristics that may make them vulnerable to being bullied (Botella, 2019; Köver & Reuter, 2019; Rauchberg, 2022). As such, the app's brief history has been riddled with evidence of the algorithmic suppression of certain groups. Other leaked information about the company revealed the tactics that TikTok's parent company, ByteDance, has used to moderate, suppress, or censor content that goes against China's foreign policy goals (Hern, 2019). This evidence shows a moderation of content about Tiananmen Square and Tibetan independence (Hern, 2019) and substantiates accusations against the company for censoring content about the Hong Kong protests (Harwell & Romm, 2019). Another research article about shadowbanning marginalized individuals underscores how more vulnerable communities are being disproportionately targeted by these moderation techniques on- and offline via Instagram (Middlebrook, 2020). This piece considered the implications that marginalized communities face being shadowbanned given the lack of awareness that users generally have about moderation algorithms. Unfortunately, the lack of guidelines provided to users to know when or why they are being shadowbanned and the lack of an appeals process to reverse the shadowban make this a particularly unjust penalization. Censorship and surveillance adversely affect users in marginalized communities and activists (Asher-Schapiro, 2022) who rely on social media to organize (Alimardani, 2022; Sandoval-Almazan & Ramon Gil-Garcia, 2014). For activists and individuals in marginalized communities, this online oppression can be the difference-maker for their organizing efforts and information sharing to be successful. The lack of specificity that social media companies offer around these practices and the lack of accuracy that social media content moderation algorithms can have when moderating the content of marginalized individuals has major implications on users' freedom of speech. The lack of transparency into moderation algorithms and recent changes to social media sites (e.g., Elon Musk purchasing Twitter; Asher-Schapiro, 2022) have prompted more discourse regarding the way social media sites moderate conversations (see Gerrard, 2018) and who are actually being moderated (Middlebrook, 2020). These systemic inadequacies of moderation algorithms are exacerbated by the common practices of most sites like these to depend on their users to report content as offensive or inappropriate instead of detecting the content through other methods (Chen et al., 2012). Relying on and entrusting users to report content becomes a greater issue of suppressing marginalized, guideline-adherent content. Users could exploit their privileges to report content for removal regardless of whether the content adheres to the site's community guidelines. Existing research about partisan-motivated behaviors like distributing fake news articles (Osmundsen et al., 2021) and labeling news as fake when it disconfirms partisans' existing beliefs (Kahne & Bowyer, 2017) offers support for the possibility that social media users may use features to report content for removal when it contradicts their attitudes. As such, activists may be up against a double bind where the automated content moderation systems in place may unfairly and inaccurately remove their content automatically and where individuals who disagree with them may flag their content to these inequitable automated systems for removal. This double bind puts activists who use social media at a disadvantage in their efforts to share their messages, despite social media becoming increasingly more necessary to engage in activism (Cammaerts, 2015). This section will discuss two recent and important examples of social movements that used social media in unique and important ways and the effect that content takedowns had or could have had on the movements. The BLM movement originated with the social media hashtag #BlackLivesMatter after George Zimmerman, the man who shot and killed Trayvon Martin in 2012, was acquitted. The hashtag and movement gained tremendous momentum in the United States in 2014 after Michael Brown in Missouri and Eric Garner in New York were killed in incidents of police brutality (Howard University School of Law, 2023). The hashtag and movement spread globally after the police murder of George Floyd in Minnesota in 2020 (Howard University School of Law, 2023; Tilly et al., 2019). In a study about collegiate athlete activism in the United States, collegiate athlete activists considered social media as one of the primary modalities through which activism occurs and through which activists can reach, amplify, and engage with messages from other members of their movements (Feder et al., 2023). For some, demonstrations took place exclusively on social media and for others, social media was an integral component of amplifying the calls to action that they publicized. These activists' organizing efforts manifested in players' strikes, donation drives after riots in the wake of the murder of George Floyd, voting campaigns, and even sales of BLM masks, the proceeds from which were donated to charities geared toward helping marginalized communities (Feder et al., 2023). Censoring the messages of these activists during their organizing efforts and during attempts to mobilize becomes an issue of political freedom as well as free speech. These experiences of suppressing political freedom and free speech were voiced by BLM activists who used TikTok during the height of BLM demonstrations that came about after the murder of George Floyd (McCluskey, 2020). These activists described drastic reductions of appearances that their videos had on users' “For You” pages on TikTok (McCluskey, 2020; Nicholas, 2022). After becoming a central site for sharing activism content (Janfaza, 2020), TikTok released a statement apologizing to its community of Black creators and users who had previously voiced feelings of marginalization and suppression on the app (see Pappas & Chikumbu, 2020). According to the official statement released by TikTok, Black content creators and allies who are users of the app changed their profile pictures and spoke out through the platform to discuss their experiences of marginalization on TikTok (Pappas & Chikumbu, 2020). The statement went on to explain that “a technical glitch made it temporarily appear as if posts uploaded using #BlackLivesMatter and #GeorgeFloyd would receive 0 views” (para. 3). Unfortunately, almost 2 months after the release of this statement, Black TikTok creators voiced continued experiences of content suppression and marginalization on the app (McCluskey, 2020). BLM activists who use the app reported not only lacking visibility to other #BlackLivesMatter content on their personalized feeds, but racist and offensive comments shared by individuals who were shown their BLM content (Allyn, 2020). Black creators who returned to posting non-BLM content on TikTok found that their regular content was receiving far less engagement (McCluskey, 2020). Further, one Black creator, Emily Barbour, recounted to TIME Magazine having created a post from a screen recording that she took containing another TikTok video in which the creator appeared in blackface in an effort to call out the video (McCluskey, 2020). While Barbour's video was hidden for being deemed a copyright violation, TikTok did not consider the video featuring a person in blackface as violating its community guidelines, leaving the video up for two more days despite thousands of users having flagged the video (McCluskey, 2020). Barbour shared with TIME the experience of racial bias and unfair treatment on TikTok (McCluskey, 2020). TikTok is not the only social media platform with documented instances of suppressing content from BLM activists. Louiza Doran is a antiracism activist and educator who experienced censorship on Instagram (Silverman, 2020). According to Doran in an interview with BuzzFeed News, her account was prohibited from livestreaming and some of her posts and comments were removed altogether (Silverman, 2020). Having been described by a spokesperson from Facebook as a technical issue that resulted in Doran's Facebook and Instagram accounts being flagged as spam (Silverman, 2020), the explanation of this issue bears striking resemblance to TikTok's “technical glitch” that caused BLM-related posts to be displayed differently (Pappas & Chikumbu, 2020). This common thread of social media companies blaming technical bugs as reasons for suppressing and censoring content points to: (1) the shortcomings of algorithmic content moderation, (2) the lack of visibility that social media companies have into the downstream effects of these moderation systems, and (3) the scapegoating strategy that these platforms use to dismiss concerns of systemic moderation bias as technological difficulty. For activists and marginalized individuals, these are not just setbacks but critical obstacles that encumber not only these people, but their followers, and, thus, factions of entire movements. The Mahsa Amini protests in Iran, which have sparked the largest antiregime uprising the country has seen since the Iranian Revolution in 1979 (France-Presse, 2022), have relied on features like hashtags and encryption on platforms like Instagram and Twitter to spread awareness and amplify messages of the protests within and outside of Iran (Amidi, 2022; Kenyon, 2023; Kumar, 2022). These antiregime protests began in 2022 in Iran and have continued as of September 2023 (Kenyon, 2023), and messages of this movement have spread across the world. Activists have been forced to rapidly transition between social media sites to counteract the Iranian government's efforts to suppress the movement's messages (Amidi, 2022; Iran International, 2022; Kumar, 2022). Due to the Iranian government's response to suppress the movement and its actors with violence, censorship of social media posts, and internet outages (CNN Special Report, 2023; France-Presse, 2022), Iranian protesters have had to depend on individuals outside of the country as well as innovations like encryption and offsite servers to share protest information on social media and to stay up-to-date (Amidi, 2022; Butcher, 2022; Kumar, 2022). Amidst the governmental censorship and violence against protestors (Amidi, 2022; CNN Special Report, 2023), the movement and its members have demonstrated resilience by quickly pivoting between social media platforms and by amplifying its messages through their online and offline presence (Kumar, 2022). As such, social media has played a critical role in the movement's sustained impact (Amidi, 2022; Kumar, 2022). Specifically, Instagram has become one of Iran's most popular social media platforms and it is the only uncensored foreign form of social media (Alimardani, 2022; Dagres, 2021). However, a policy change that Meta made in 2022 in response to the Russian invasion of Ukraine has led to the removal of large amounts of Iranian protest-related content that contained a common Iranian protest slogan (Alimardani, 2022). This slogan, which translates into English as calls for death to the military forces, current supreme leader, and current president, exist culturally as a symbolic dissident call against Iran's authoritarian regime rather than a true call for violence (Alimardani, 2022). Where Meta used to include these slogans in their exceptions of community guidelines, a new policy change that Meta made rolls back these exceptions both for these slogans in Iran and for other protest slogans used in Ukraine during the Russian invasion (Alimardani, 2022; Biddle, 2022; Vengattil & Culliford, 2022). As such, Meta has been criticized for prioritizing more respect for the dictators than for the protestors (Höppner, 2022) where their policies regarding human rights and free speech more align with U.S. policy (Biddle, 2022). For Iranian protesters, their government's capabilities paired with the policy decisions that social media companies like Meta make in moderating their content pose a unique combined challenge for organizing and mobilizing. This mass systematic removal of posts from personal accounts and important national media outlets that came about without warning in the wake of Meta's policy change (Alimardani, 2022) has exacerbated the extent to which these protesters are forced to monitor their posts and strategize their technology use. Further, evidence of the Iranian government hacking activists' accounts or taking control of the accounts of protesters in their custody (Polglase & Mezzofiore, 2022) pose significant threats to activists and to their movement. Indeed, social media platforms' moderation policies and practices have important implications on activists and their movements. This Meta content moderation policy change has produced a similar outcome to the censorship practiced by the Iranian government. As such, this situation speaks to the necessity for social media platforms to identify and institute culturally nuanced understanding in their moderation policies and practices if their objectives truly involve promoting free speech on their sites. In addition to the concerns that algorithmic moderation systems perpetuate systemic political injustices that persist offline, these practices come with numerous ethical concerns and issues of scale (Gorwa et al., 2020; Marcondes et al., 2021). Therefore, the following sections will review political and ethical concerns and recommendations specifically as these algorithmic moderation practices pertain to activism and organizing. There are many political implications of the current structure of algorithmic content moderation that should be considered. Because social media companies' content moderation algorithms' display of bias against marginalized individuals has been documented (Middlebrook, 2020; Rauchberg, 2022) and because activists are often individuals who identify with at least one marginalized group (Heaney, 2022; Kluch, 2022), erroneously removing content shared by individuals who are activists and who are marginalized exacerbates the marginalization (Middlebrook, 2020). Societally, this practice stunts movements, hampers representation, targets groups, silences individuals, violates rights to free speech, and multiplies the workload of the content creators who identify in these ways and experience this mistreatment. In addition to the specific issues that are considered in activists' posts, these individual implications carry weight for political engagement, political information seeking, and public policy. Assumptions of western ideals such as freedom of speech and freedom of the press underlie the discussion in this paper. As polarization across party lines on sociopolitical issues continues to grow, partisan behaviors such as sharing news articles that disparage opposing political parties and party members on social media have persisted (Osmundsen et al., 2021). Of course, reporting content is now feasibly emerging as another possible partisan behavior that works to disparage opposing perspectives by flagging them for removal. However, these behaviors are not only centralized around western countries where apparent government censorship may not be present because censorship may come from sources other than government entities (e.g., host social media companies or third parties). For activists in countries where government censorship of social media content is exerted (see Casilli & Tubaro, 2012; Golovchenko, 2022; Liu & Zhao, 2021; Poell, 2014; Tai & Fu, 2020), activists' labor is multiplied to both create and publicize their messages, mobilize individuals, and replicate this process to combat either the algorithmic content removal or governmental censorship. Because the individuals who create and deploy algorithms are embedded in systems of oppression, their products reproduce these systems (Middlebrook, 2020), which ultimately oppresses activists and marginalized individuals, thereby oppressing social movements and organizing processes, political discourse, political engagement, and, thus, political outcomes. As such, even activists who reside in countries that tout the prioritization of free speech are experiencing instances of censorship (Langvardt, 2018). Instances like these where certain perspectives are excluded from discourse have been shown to impact others' perceptions of public opinion and marginalize the expression of those with similar positions (Gearhart & Zhang, 2015; Roy, 2017). Ethical concerns for algorithmic content moderation practices involve democratizing content removal features, the relational and emotional ramifications of these systems, and the lack of cultural and linguistic understanding that these practices demonstrate. Enabling all users to report content for removal broadens the potential for innocent content to be removed automatically by these moderation algorithms as it puts the power in users' hands, thereby increasing the amount of flagged content that is appropriate and, thus, the number of ambiguous cases that human moderators would potentially need to review. Though social media companies have focused on effectively reducing manual moderation while keeping hate speech and cyberbullying at bay, the overreach of these algorithms relative to the inaccuracy of these moderation algorithms has not been considered (Center for Countering Digital Hate, 2022). However, existing research has also documented relational and emotional ramifications that individuals who report having been censored and shadowbanned experience online and offline (West, 2018). What many may not also recognize is the emotional toll that is placed on both human moderators who work for social media companies (Gillespie, 2018, 2020) and those who consume content that should be removed and was not detected by algorithms (Center for Countering Digital Hate, 2022). It is critical to recognize the emotional labor that human content moderators tasked with categorizing content into appropriate content and inappropriate content in the building of training datasets are confronted with (Gillespie, 2018; Sinnreich, 2018). Further, although human content moderators can identify ironic statements and greater nuance than moderation algorithms (Duarte et al., 2017), human content moderation applies the same level of bias to what is considered hate speech or inappropriate content with potentially more inconsistency. Given the socially constructed nature of hate speech and what it is or is not, the notion of hate speech may differ culturally and cannot be fixed to a singular point in time (Gillespie, 2020). Meta's content moderation policy change that led to the mass removal of posts using Iran's dissident slogan is an important example of this (Alimardani, 2022). Meta has also demonstrated an incapacity for moderating content in non-Western languages and applying Western ideals to moderation policies in other parts of the world (Alimardani & Elswah, 2021; Fatafta, 2022). Thus, Meta has demonstrated a lack of cultural sensitivity in its creation and institution of moderation policy and practices. These issues, however, may be ameliorated by using moderation algorithms to flag the most egregious and universally held forms of offensive speech and promoting more ambiguous posts to human coders (Gillespie, 2020). This change and increased transparency of social media companies about the moderation algorithms they deploy and content removal for users are meaningful developments that social media platforms could offer to help wrongfully censored content be corrected. Although each of these implications are significant, issues such as the lack of transparency into the innerworkings of social media moderation algorithms and the lack of structural opportunities to appeal or inquire about moderation practices are some of the most integral. Despite the fact that experts in AI have developed ethics guidelines for users and for developers (see Ryan & Stahl [2020] for one example), social media platforms such as Instagram or Twitter do not seem to publicize or practice ethical competencies that guide their algorithm development and implementation. In fact, with Elon Musk's purchase of Twitter, the team dedicated to ethically practicing AI and promoting transparency and fairness was reportedly disbanded (Knight, 2022). Social media companies should also broaden their organizational structure so that developers institute moderation practices specific to the regions and languages of which they demonstrate sufficient cultural understanding. By offering more transparent affordances such as notifications and appeals features for users whose content is removed or censored from searches, users can be made aware of how their content is violating regulations and appeal these decisions. Because Twitter, Instagram, and Facebook have notifications and appeals options in place for other instances, these companies already have the infrastructure that could be extended to allow for these possibilities. Further, algorithms could be updated as coders review appeals in an effort to protect and better represent the groups of people like activists and marginalized individuals whose content tends to be removed more often and unjustly. Additional features could involve implementing a business profile category for users to label their accounts as political activists. Social media companies could then vet the profiles and institute exemptions in their content moderation algorithms specifically for accounts with this profile category type. The first goal of this paper was to contribute to the academic discourse sharing more information about how certain individuals are being censored and silenced by social content moderation algorithms. To do this, this paper considered the various forms of social media content moderation along with how these features work in tension with free speech and equal treatment of user groups that are espoused by these platforms. The second goal of this paper was to propose for social media companies to implement greater transparency into their content moderation practices. This paper recommends extending their current appeals infrastructure to notify users when their content or accounts are being hidden from searches or public pages, changing the structure of algorithmic content moderation, and offering additional features to make moderation more accurate. By implementing algorithms for more extreme, offensive content, enlisting human moderators to code more ambiguous, nuanced posts (Gillespie, 2020) and building algorithms to be more iterative so that they can be updated as coders review appeals to automated moderation decisions, moderation algorithms will have a much greater capacity for ethically, accurately, and effectively moderating hate speech while promoting free speech equally. Social media companies have a way to go before arriving at a finish line for ethically, accurately, and effectively mitigating hate speech and encouraging political and social discourse. Major recurring issues such as the lack of transparency of the innerworkings of social media algorithmic moderation systems and the lack of structural opportunities to appeal or inquire about moderation practices are only the tip of the iceberg. As marginalized communities are targeted through automated moderation practices like shadowbanning, oppressive and inequitable treatment of these users is reinforced online while offline hegemonic systems that silence marginalized communities grow stronger. The issue of social media content moderation—particularly algorithmic content takedowns—and how they affect social activists who are using social media as a tool for organizing is of significance both for social justice movements and their members as well as social media platforms and their stakeholders. As cyberactivism and the online amplification of offline social issues integrate with physical mobilizing (Sandoval-Almazan & Ramon Gil-Garcia, 2014), the role of social media in supporting social activists' online communication is increasingly important. Equipping all users, regardless of the extreme ideological positions that they may carry or vocalize, with these tools further exacerbates the gap between offensive or inappropriate content and removed content. Doing so runs counter to the notion of free speech, thereby making this capability an issue of public policy. Indeed, making features like this widely accessible does more to reinforce the offline hegemony online and contributes more to engaging in free content flagging than it does to engaging in free speech.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
8.40
自引率
10.20%
发文量
51
期刊介绍: Understanding public policy in the age of the Internet requires understanding how individuals, organizations, governments and networks behave, and what motivates them in this new environment. Technological innovation and internet-mediated interaction raise both challenges and opportunities for public policy: whether in areas that have received much work already (e.g. digital divides, digital government, and privacy) or newer areas, like regulation of data-intensive technologies and platforms, the rise of precarious labour, and regulatory responses to misinformation and hate speech. We welcome innovative research in areas where the Internet already impacts public policy, where it raises new challenges or dilemmas, or provides opportunities for policy that is smart and equitable. While we welcome perspectives from any academic discipline, we look particularly for insight that can feed into social science disciplines like political science, public administration, economics, sociology, and communication. We welcome articles that introduce methodological innovation, theoretical development, or rigorous data analysis concerning a particular question or problem of public policy.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信