{"title":"Fingerprints of Conspiracy Theories: Identifying Signature Information Sources of a Misleading Narrative and Their Roles in Shaping Message Content and Dissemination","authors":"Soojong Kim, Kwanho Kim, Haoning Xue","doi":"10.54501/jots.v2i2.152","DOIUrl":"https://doi.org/10.54501/jots.v2i2.152","url":null,"abstract":"This study investigates the role of information sources in the propagation and reception of misleading narratives on social media, focusing on the case of the Chemtrail conspiracy theory—a false claim that the trails in the sky behind airplanes are chemicals deliberately spread for sinister reasons. We collected data from Facebook Pages and Groups discussing the conspiracy theory. We specifically focused on identifying and analyzing “signature” information sources, which are repeatedly used by online communities engaged in the discussion of a misleading narrative but are not widely used by other communities. The findings indicate that messages referencing signature sources contain more death-, illness-, risk-, and health-related words, convey more negativity, and elicit more negative reactions from users, compared with those without signature sources. The insights from this study could contribute to the development of effective strategies to monitor and counter the spread of misleading narratives in digital spaces.","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"85 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140422799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Naomi Shiffman, Carly Miller, Manuel Parra, Claudia Flores-Saviaga
{"title":"Burden of Proof: Lessons Learned for Regulators from the Oversight Board’s Implementation Work","authors":"Naomi Shiffman, Carly Miller, Manuel Parra, Claudia Flores-Saviaga","doi":"10.54501/jots.v2i2.168","DOIUrl":"https://doi.org/10.54501/jots.v2i2.168","url":null,"abstract":"This paper examines the Oversight Board’s work in tracking and independently verifying Meta’s implementation of nonbinding recommendations, with the aim of supporting the development of best practices for social media auditing under emerging regulation. While the Board only considers decisions made by Meta, the lessons learned may be useful for regulators and other social media platforms. The Oversight Board is an experimental governance body created by Meta in 2020 to make independent policy decisions and recommendations that address the most significant and difficult challenges on its platforms. Its goal is to ensure users’ rights and interests by bringing greater transparency, consistency","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"69 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140418076","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Securing Federated Platforms","authors":"Yoel Roth, Samantha Lai","doi":"10.54501/jots.v2i2.171","DOIUrl":"https://doi.org/10.54501/jots.v2i2.171","url":null,"abstract":"As the social media landscape undergoes broad transformation for the first time in over a decade, with alternative platforms like Mastodon, Bluesky, and Threads emerging where X has receded, many users and observers have celebrated the promise of these new services and their visions of alternative governance structures that empower consumers. Drawing on a large-scale textual analysis of platform moderation policies, capabilities, and transparency mechanisms, as well as semistructured group interviews with developers, administrators, and moderators of federated platforms, we found that federated platforms face considerable obstacles to robust and scalable governance, particularly with regard to persistent threats such as coordinated behavior and spam. Key barriers identified include underdeveloped moderation technologies and a lack of sustainable financial models for trust and safety work. We offer four solutions to the collective safety and security risks identified: (1) institutionalize shared responses to critical harms, (2) build transparent governance into the system, (3) invest in open-source tooling, and (4) enable data sharing across instances.","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"22 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140420291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Effects of Browsing Conditions and Visual Alert Design on Human Susceptibility to Deepfakes","authors":"Emilie Josephs, Camilo Luciano Fosco, Aude Oliva","doi":"10.54501/jots.v2i2.144","DOIUrl":"https://doi.org/10.54501/jots.v2i2.144","url":null,"abstract":"The increasing reach of deepfakes raises practical questions about people’s ability to detect false videos online. How vulnerable are people to deepfake videos? What technologies can help improve detection? Previous experiments that measure human deepfake detection historically omit a number of conditions that can exist in typical browsing conditions. Here, we operationalized four such conditions (low prevalence, brief presentation, low video quality, and divided attention), and found in a series of online experiments that all conditions lowered detection relative to baseline, suggesting that the current literature underestimates people’s susceptibility to deepfakes. Next, we examined how AI assistance could be integrated into the human decision process. We found that a model that exposes deepfakes by amplifying artifacts increases detection rates, and also leads to higher rates of incorporating AI feedback and higher final confidence than text-based prompts. Overall, this suggests that visual indicators that cause distortions on fake videos may be effective at mitigating the impact of falsified video.","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"97 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140423796","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sarah A. Fisher, Beatriz Kira, Kiran Arabaghatta Basavaraj, Jeffrey Howard
{"title":"Should Politicians Be Exempt from Fact-Checking?","authors":"Sarah A. Fisher, Beatriz Kira, Kiran Arabaghatta Basavaraj, Jeffrey Howard","doi":"10.54501/jots.v2i2.170","DOIUrl":"https://doi.org/10.54501/jots.v2i2.170","url":null,"abstract":"","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"66 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140421452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Content Modeling in Multi-Platform Multilingual Social Media Data","authors":"Arman Setser, Libby Lange, Kyle Weiss, Vlad Barash","doi":"10.54501/jots.v2i2.136","DOIUrl":"https://doi.org/10.54501/jots.v2i2.136","url":null,"abstract":"An increase in the use of social media as the primary news source for the general population has created an ecosystem in which organic conversation commingles with inorganically seeded and amplified narratives, which can include public relations and marketing activity but also covert and malign influence operations. An efficient and easily understandable analysis of such data is important, as it allows relevant stakeholders to protect online communities and free discussion while better identifying activity and content that may violate social media platform terms of service. To accomplish this, we propose a method of large-scale social media data analysis, which allows for multilingual conversations to be analyzed in depth across any number of social media platforms simultaneously. Our method uses a text embedding model, i.e., a natural language processing model that holds semantic and contextual understandings of language. The model uses an “understanding” of language to represent posts as coordinates in a high-dimensional space, such that posts with similar meanings are assigned coordinates close together. We then cluster and analyze the posts to identify online topics of conversation existing across multiple social media platforms. We explicitly show how our method can be applied to four different datasets, three consisting of Chinese social media posts related to the Belt and Road Initiative and one relating to the Russia-Ukraine war, and we find politically-influenced conversations that contain misleading information relating to the Chinese government and the Russia-Ukraine war.","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"433 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140417156","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Bridging Theory & Practice: Examining the State of Scholarship Using the History of Trust and Safety Archive","authors":"Megan Knittel, Amanda Menking","doi":"10.54501/jots.v2i2.173","DOIUrl":"https://doi.org/10.54501/jots.v2i2.173","url":null,"abstract":"","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"119 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140421164","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Brian Timmerman, Pulak Mehta, Progga Deb, Kevin Gallagher, Brendan Dolan-Gavitt, Siddharth Garg, Rachel Greenstadt
{"title":"Studying the Online Deepfake Community","authors":"Brian Timmerman, Pulak Mehta, Progga Deb, Kevin Gallagher, Brendan Dolan-Gavitt, Siddharth Garg, Rachel Greenstadt","doi":"10.54501/jots.v2i1.126","DOIUrl":"https://doi.org/10.54501/jots.v2i1.126","url":null,"abstract":"Deepfakes have become a dual-use technology with applications in the domains of art, science, and industry. However, the technology can also be leveraged maliciously in areas such as disinformation, identity fraud, and harassment. In response to the technology's dangerous potential many deepfake creation communities have been deplatformed, including the technology's originating community – r/deepfakes. Opening in February 2018, just eight days after the removal of r/deepfakes, MrDeepFakes (MDF) went online as a privately owned platform to fulfill the role of community hub, and has since grown into the largest dedicated deepfake creation and discussion platform currently online. This position of community hub is balanced against the site's other main purpose, which is the hosting of deepfake pornography depicting public figures- produced without consent. In this paper we explore the two largest deepfake communities that have existed via a mixed methods approach utilizing quantitative and qualitative analysis. We seek to identify how these platforms were and are used by their members, what opinions these deepfakers hold about the technology and how it is seen by society at large, and identify how deepfakes-as-disinformation is viewed by the community. We find that there is a large emphasis on technical discussion on these platforms, intermixed with potentially malicious content. Additionally, we find the deplatforming of deepfake communities early in the technology's life has significantly impacted trust regarding alternative community platforms.","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136236838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Toward Better Automated Content Moderation in Low-Resource Languages","authors":"Gabriel Nicholas, Aliya Bhatia","doi":"10.54501/jots.v2i1.150","DOIUrl":"https://doi.org/10.54501/jots.v2i1.150","url":null,"abstract":"Social media companies have learned the hard way that poor moderation of content in languages other than English can have grave consequences. Leaving harmful content up, particularly in regions where social media platforms are primary news and communication channels, has fueled conspiracy theories, violence, and even genocide around the world (Fink 2018; Iyengar 2018). American social media companies have long faced criticism for underinvesting in regions outside the US and Europe. In response, Meta, Google, and others have begun to deploy newmultilingual language models that they claim can effectively detect and take action on harmful content in dozens if not hundreds of languages (Jigsaw 2021; Meta AI 2021b).","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136236857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Social Contagion, from Suicide to Online Challenges to Eating Disorders: Current Research and Harm Mitigation Strategies for Youth Online","authors":"Anjuli Corzine, Vicki Harrison","doi":"10.54501/jots.v2i1.145","DOIUrl":"https://doi.org/10.54501/jots.v2i1.145","url":null,"abstract":"The growth of socialmedia sites has rapidly changed theway that people interactwith one another online, especially for adolescents and young adults who are the primary users of social media in the United States (Carlyle et al. 2018). Ninety-six percent of teenagers report having consistent access to a smartphone and have been shown to use a larger number of social media platforms than in years past (Roth et al. 2020). In the US, 86% of people use at least one social media platform, and 85% of children 12to 17-years old report regular social media use (Carlyle et al. 2018; Chung et al. 2021). Given their frequent use of social media, of specific concern are sites that facilitate the promotion of risky behaviors. For example, viral trends on platforms like TikTok and Twitter, in which users are encouraged to engage in non-normative behaviors, are increasingly common, some of them promoting the performance of risky acts (Abraham et al. 2022). In addition to potentially dangerous trends, there has been a rise of online communities dedicated to the promotion of self-harming behaviors that offer community, tips, and support in continuing to participate in damaging behavior (Knapton 2013).","PeriodicalId":477175,"journal":{"name":"Journal of online trust and safety","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136236987","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}