François t'Serstevens, Roberto Cerina, Giulia Piccillo
{"title":"Fake News Detection via Wisdom of Synthetic & Representative Crowds","authors":"François t'Serstevens, Roberto Cerina, Giulia Piccillo","doi":"arxiv-2408.03154","DOIUrl":"https://doi.org/arxiv-2408.03154","url":null,"abstract":"Social media companies have struggled to provide a democratically legitimate\u0000definition of \"Fake News\". Reliance on expert judgment has attracted criticism\u0000due to a general trust deficit and political polarisation. Approaches reliant\u0000on the ``wisdom of the crowds'' are a cost-effective, transparent and inclusive\u0000alternative. This paper provides a novel end-to-end methodology to detect fake\u0000news on X via \"wisdom of the synthetic & representative crowds\". We deploy an\u0000online survey on the Lucid platform to gather veracity assessments for a number\u0000of pandemic-related tweets from crowd-workers. Borrowing from the MrP\u0000literature, we train a Hierarchical Bayesian model to predict the veracity of\u0000each tweet from the perspective of different personae from the population of\u0000interest. We then weight the predicted veracity assessments according to a\u0000representative stratification frame, such that decisions about ``fake'' tweets\u0000are representative of the overall polity of interest. Based on these aggregated\u0000scores, we analyse a corpus of tweets and perform a second MrP to generate\u0000state-level estimates of the number of people who share fake news. We find\u0000small but statistically meaningful heterogeneity in fake news sharing across US\u0000states. At the individual-level: i. sharing fake news is generally rare, with\u0000an average sharing probability interval [0.07,0.14]; ii. strong evidence that\u0000Democrats share less fake news, accounting for a reduction in the sharing odds\u0000of [57.3%,3.9%] relative to the average user; iii. when Republican definitions\u0000of fake news are used, it is the latter who show a decrease in the propensity\u0000to share fake news worth [50.8%, 2.0%]; iv. some evidence that women share less\u0000fake news than men, an effect worth a [29.5%,4.9%] decrease.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"89 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141930924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reasons to Doubt the Impact of AI Risk Evaluations","authors":"Gabriel Mukobi","doi":"arxiv-2408.02565","DOIUrl":"https://doi.org/arxiv-2408.02565","url":null,"abstract":"AI safety practitioners invest considerable resources in AI system\u0000evaluations, but these investments may be wasted if evaluations fail to realize\u0000their impact. This paper questions the core value proposition of evaluations:\u0000that they significantly improve our understanding of AI risks and,\u0000consequently, our ability to mitigate those risks. Evaluations may fail to\u0000improve understanding in six ways, such as risks manifesting beyond the AI\u0000system or insignificant returns from evaluations compared to real-world\u0000observations. Improved understanding may also not lead to better risk\u0000mitigation in four ways, including challenges in upholding and enforcing\u0000commitments. Evaluations could even be harmful, for example, by triggering the\u0000weaponization of dual-use capabilities or invoking high opportunity costs for\u0000AI safety. This paper concludes with considerations for improving evaluation\u0000practices and 12 recommendations for AI labs, external evaluators, regulators,\u0000and academic researchers to encourage a more strategic and impactful approach\u0000to AI risk assessment and mitigation.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"24 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141930926","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Viktor Suter, Charles Ma, Gina Poehlmann, Miriam Meckel, Lea Steinacker
{"title":"An integrated view of Quantum Technology? Mapping Media, Business, and Policy Narratives","authors":"Viktor Suter, Charles Ma, Gina Poehlmann, Miriam Meckel, Lea Steinacker","doi":"arxiv-2408.02236","DOIUrl":"https://doi.org/arxiv-2408.02236","url":null,"abstract":"Narratives play a vital role in shaping public perceptions and policy on\u0000emerging technologies like quantum technology (QT). However, little is known\u0000about the construction and variation of QT narratives across societal domains.\u0000This study examines how QT is presented in business, media, and government\u0000texts using thematic narrative analysis. Our research design utilizes an\u0000extensive dataset of 36 government documents, 165 business reports, and 2,331\u0000media articles published over 20 years. We employ a computational social\u0000science approach, combining BERTopic modeling with qualitative assessment to\u0000extract themes and narratives. The findings show that public discourse on QT\u0000reflects prevailing social and political agendas, focusing on technical and\u0000commercial potential, global conflicts, national strategies, and social issues.\u0000Media articles provide the most balanced coverage, while business and\u0000government discourses often overlook societal implications. We discuss the\u0000ramifications for integrating QT into society and the need for wellinformed\u0000public discourse.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"46 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141930923","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jacques P. Fleischer, Ryan Pallack, Ahan Mishra, Gustavo Riente de Andrade, Subhadipto Poddar, Emmanuel Posadas, Robert Schenck, Tania Banerjee, Anand Rangarajan, Sanjay Ranka
{"title":"Video-based Pedestrian and Vehicle Traffic Analysis During Football Games","authors":"Jacques P. Fleischer, Ryan Pallack, Ahan Mishra, Gustavo Riente de Andrade, Subhadipto Poddar, Emmanuel Posadas, Robert Schenck, Tania Banerjee, Anand Rangarajan, Sanjay Ranka","doi":"arxiv-2408.02146","DOIUrl":"https://doi.org/arxiv-2408.02146","url":null,"abstract":"This paper utilizes video analytics to study pedestrian and vehicle traffic\u0000behavior, focusing on analyzing traffic patterns during football gamedays. The\u0000University of Florida (UF) hosts six to seven home football games on Saturdays\u0000during the college football season, attracting significant pedestrian activity.\u0000Through video analytics, this study provides valuable insights into the impact\u0000of these events on traffic volumes and safety at intersections. Comparing\u0000pedestrian and vehicle activities on gamedays versus non-gamedays reveals\u0000differing patterns. For example, pedestrian volume substantially increases\u0000during gamedays, which is positively correlated with the probability of the\u0000away team winning. This correlation is likely because fans of the home team\u0000enjoy watching difficult games. Win probabilities as an early predictor of\u0000pedestrian volumes at intersections can be a tool to help traffic professionals\u0000anticipate traffic management needs. Pedestrian-to-vehicle (P2V) conflicts\u0000notably increase on gamedays, particularly a few hours before games start.\u0000Addressing this, a \"Barnes Dance\" movement phase within the intersection is\u0000recommended. Law enforcement presence during high-activity gamedays can help\u0000ensure pedestrian compliance and enhance safety. In contrast, we identified\u0000that vehicle-to-vehicle (V2V) conflicts generally do not increase on gamedays\u0000and may even decrease due to heightened driver caution.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"57 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141930998","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Robert Wolfe, Aayushi Dangol, Alexis Hiniker, Bill Howe
{"title":"Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI","authors":"Robert Wolfe, Aayushi Dangol, Alexis Hiniker, Bill Howe","doi":"arxiv-2408.01959","DOIUrl":"https://doi.org/arxiv-2408.01959","url":null,"abstract":"Multimodal AI models capable of associating images and text hold promise for\u0000numerous domains, ranging from automated image captioning to accessibility\u0000applications for blind and low-vision users. However, uncertainty about bias\u0000has in some cases limited their adoption and availability. In the present work,\u0000we study 43 CLIP vision-language models to determine whether they learn\u0000human-like facial impression biases, and we find evidence that such biases are\u0000reflected across three distinct CLIP model families. We show for the first time\u0000that the the degree to which a bias is shared across a society predicts the\u0000degree to which it is reflected in a CLIP model. Human-like impressions of\u0000visually unobservable attributes, like trustworthiness and sexuality, emerge\u0000only in models trained on the largest dataset, indicating that a better fit to\u0000uncurated cultural data results in the reproduction of increasingly subtle\u0000social biases. Moreover, we use a hierarchical clustering approach to show that\u0000dataset size predicts the extent to which the underlying structure of facial\u0000impression bias resembles that of facial impression bias in humans. Finally, we\u0000show that Stable Diffusion models employing CLIP as a text encoder learn facial\u0000impression biases, and that these biases intersect with racial biases in Stable\u0000Diffusion XL-Turbo. While pretrained CLIP models may prove useful for\u0000scientific studies of bias, they will also require significant dataset curation\u0000when intended for use as general-purpose models in a zero-shot setting.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"158 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141931001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ML-EAT: A Multilevel Embedding Association Test for Interpretable and Transparent Social Science","authors":"Robert Wolfe, Alexis Hiniker, Bill Howe","doi":"arxiv-2408.01966","DOIUrl":"https://doi.org/arxiv-2408.01966","url":null,"abstract":"This research introduces the Multilevel Embedding Association Test (ML-EAT),\u0000a method designed for interpretable and transparent measurement of intrinsic\u0000bias in language technologies. The ML-EAT addresses issues of ambiguity and\u0000difficulty in interpreting the traditional EAT measurement by quantifying bias\u0000at three levels of increasing granularity: the differential association between\u0000two target concepts with two attribute concepts; the individual effect size of\u0000each target concept with two attribute concepts; and the association between\u0000each individual target concept and each individual attribute concept. Using the\u0000ML-EAT, this research defines a taxonomy of EAT patterns describing the nine\u0000possible outcomes of an embedding association test, each of which is associated\u0000with a unique EAT-Map, a novel four-quadrant visualization for interpreting the\u0000ML-EAT. Empirical analysis of static and diachronic word embeddings, GPT-2\u0000language models, and a CLIP language-and-image model shows that EAT patterns\u0000add otherwise unobservable information about the component biases that make up\u0000an EAT; reveal the effects of prompting in zero-shot models; and can also\u0000identify situations when cosine similarity is an ineffective metric, rendering\u0000an EAT unreliable. Our work contributes a method for rendering bias more\u0000observable and interpretable, improving the transparency of computational\u0000investigations into human minds and societies.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141931000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Drama Machine: Simulating Character Development with LLM Agents","authors":"Liam Magee, Vanicka Arora, Gus Gollings, Norma Lam-Saw","doi":"arxiv-2408.01725","DOIUrl":"https://doi.org/arxiv-2408.01725","url":null,"abstract":"This paper explores use of multiple large language model (LLM) agents to\u0000simulate complex, dynamic characters in dramatic scenarios. We introduce a\u0000`drama machine' framework that coordinates interactions between LLM agents\u0000playing different `Ego' and `Superego' psychological roles. In roleplay\u0000simulations, this design allows intersubjective dialogue and intra-subjective\u0000internal monologue to develop in parallel. We apply this framework to two\u0000dramatic scenarios - an interview and a detective story - and compare character\u0000development with and without the Superego's influence. Though exploratory,\u0000results suggest this multi-agent approach can produce more nuanced, adaptive\u0000narratives that evolve over a sequence of dialogical turns. We discuss\u0000different modalities of LLM-based roleplay and character development, along\u0000with what this might mean for conceptualization of AI subjectivity. The paper\u0000concludes by considering how this approach opens possibilities for thinking of\u0000the roles of internal conflict and social performativity in AI-based\u0000simulation.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"51 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141930925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Advancing Mental Health Pre-Screening: A New Custom GPT for Psychological Distress Assessment","authors":"Jinwen Tang, Yi Shang","doi":"arxiv-2408.01614","DOIUrl":"https://doi.org/arxiv-2408.01614","url":null,"abstract":"This study introduces 'Psycho Analyst', a custom GPT model based on OpenAI's\u0000GPT-4, optimized for pre-screening mental health disorders. Enhanced with\u0000DSM-5, PHQ-8, detailed data descriptions, and extensive training data, the\u0000model adeptly decodes nuanced linguistic indicators of mental health disorders.\u0000It utilizes a dual-task framework that includes binary classification and a\u0000three-stage PHQ-8 score computation involving initial assessment, detailed\u0000breakdown, and independent assessment, showcasing refined analytic\u0000capabilities. Validation with the DAIC-WOZ dataset reveals F1 and Macro-F1\u0000scores of 0.929 and 0.949, respectively, along with the lowest MAE and RMSE of\u00002.89 and 3.69 in PHQ-8 scoring. These results highlight the model's precision\u0000and transformative potential in enhancing public mental health support,\u0000improving accessibility, cost-effectiveness, and serving as a second opinion\u0000for professionals.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"108 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141930927","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hai Yang, Hongying Wu, Lauren Whang, Xiyuan Ren, Joseph Y. J. Chow
{"title":"Welfare, sustainability, and equity evaluation of the New York City Interborough Express using spatially heterogeneous mode choice models","authors":"Hai Yang, Hongying Wu, Lauren Whang, Xiyuan Ren, Joseph Y. J. Chow","doi":"arxiv-2408.01562","DOIUrl":"https://doi.org/arxiv-2408.01562","url":null,"abstract":"The Metropolitan Transit Authority (MTA) proposed building a new light rail\u0000route called the Interborough Express (IBX) to provide a direct, fast transit\u0000linkage between Queens and Brooklyn. An open-access synthetic citywide trip\u0000agenda dataset and a block-group-level mode choice model are used to assess the\u0000potential impact IBX could bring to New York City (NYC). IBX could save 28.1\u0000minutes to potential riders across the city. For travelers either going to or\u0000departing from areas close to IBX, the average time saving is projected to be\u000029.7 minutes. IBX is projected to have more than 254 thousand daily ridership\u0000after its completion (69% higher than reported in the official IBX proposal).\u0000Among those riders, more than 78 thousand people (30.8%) would come from\u0000low-income households while 165 thousand people (64.7%) would start or end\u0000along the IBX corridor. The addition of IBX would attract more than 50 thousand\u0000additional daily trips to transit mode, among which more than 16 thousand would\u0000be switched from using private vehicles, reducing potential greenhouse gas\u0000(GHG) emissions by 29.28 metric tons per day. IBX can also bring significant\u0000consumer surplus benefits to the communities, which are estimated to be $1.25\u0000USD per trip, or as high as $1.64 per trip made by a low-income traveler. While\u0000benefits are proportionately higher for lower-income users, the service does\u0000not appear to significantly reduce the proportion of travelers whose consumer\u0000surpluses fall below 10% of the population average (already quite low).","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"194 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141930997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Victor Borza, Andrew Estornell, Ellen Wright Clayton, Chien-Ju Ho, Russell Rothman, Yevgeniy Vorobeychik, Bradley Malin
{"title":"Adaptive Recruitment Resource Allocation to Improve Cohort Representativeness in Participatory Biomedical Datasets","authors":"Victor Borza, Andrew Estornell, Ellen Wright Clayton, Chien-Ju Ho, Russell Rothman, Yevgeniy Vorobeychik, Bradley Malin","doi":"arxiv-2408.01375","DOIUrl":"https://doi.org/arxiv-2408.01375","url":null,"abstract":"Large participatory biomedical studies, studies that recruit individuals to\u0000join a dataset, are gaining popularity and investment, especially for analysis\u0000by modern AI methods. Because they purposively recruit participants, these\u0000studies are uniquely able to address a lack of historical representation, an\u0000issue that has affected many biomedical datasets. In this work, we define\u0000representativeness as the similarity to a target population distribution of a\u0000set of attributes and our goal is to mirror the U.S. population across\u0000distributions of age, gender, race, and ethnicity. Many participatory studies\u0000recruit at several institutions, so we introduce a computational approach to\u0000adaptively allocate recruitment resources among sites to improve\u0000representativeness. In simulated recruitment of 10,000-participant cohorts from\u0000medical centers in the STAR Clinical Research Network, we show that our\u0000approach yields a more representative cohort than existing baselines. Thus, we\u0000highlight the value of computational modeling in guiding recruitment efforts.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"30 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141931006","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}