Social Science Computer Review最新文献

The Efficacy of Large Language Models and Crowd Annotation for Accurate Content Analysis of Political Social Media Messages 大型语言模型和人群注释对政治社交媒体信息准确内容分析的功效

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-05-02 DOI: 10.1177/08944393251334977

Jennifer Stromer-Galley, Brian McKernan, Saklain Zaman, Chinmay Maganur, Sampada Regmi

{"title":"The Efficacy of Large Language Models and Crowd Annotation for Accurate Content Analysis of Political Social Media Messages","authors":"Jennifer Stromer-Galley, Brian McKernan, Saklain Zaman, Chinmay Maganur, Sampada Regmi","doi":"10.1177/08944393251334977","DOIUrl":"https://doi.org/10.1177/08944393251334977","url":null,"abstract":"Systematic content analysis of messaging has been a staple method in the study of communication. While computer-assisted content analysis has been used in the field for three decades, advances in machine learning and crowd-based annotation combined with the ease of collecting volumes of text-based communication via social media have made the opportunities for classification of messages easier and faster. The greatest advancement yet might be in the form of general intelligence large language models (LLMs), which are ostensibly able to accurately and reliably classify messages by leveraging context to disambiguate meaning. It is unclear, however, how effective LLMs are in deploying the method of content analysis. In this study, we compare the classification of political candidate social media messages between trained annotators, crowd annotators, and large language models from Open AI accessed through the free Web (ChatGPT) and the paid API (GPT API) on five different categories of political communication commonly used in the literature. We find that crowd annotation generally had higher F1 scores than ChatGPT and an earlier version of the GPT API, although the newest version, GPT-4 API, demonstrated good performance as compared with the crowd and with ground truth data derived from trained student annotators. This study suggests the application of any LLM to an annotation task requires validation, and that freely available and older LLM models may not be effective for studying human communication.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"43 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143901247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Theory-Driven Approach to Fake News/Information Disorder Analysis and Explanation via Target-Based Emotion–Stance Analysis (TESA) and Interpretive Graph Generation (IGG) 基于目标情绪立场分析（TESA）和解释图生成（IGG）的假新闻/信息混乱分析与解释的理论驱动方法

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-05-02 DOI: 10.1177/08944393251338403

Xingyu Ken Chen, Jin-Cheon Na

{"title":"A Theory-Driven Approach to Fake News/Information Disorder Analysis and Explanation via Target-Based Emotion–Stance Analysis (TESA) and Interpretive Graph Generation (IGG)","authors":"Xingyu Ken Chen, Jin-Cheon Na","doi":"10.1177/08944393251338403","DOIUrl":"https://doi.org/10.1177/08944393251338403","url":null,"abstract":"Information disorder (IDO) presents a persistent challenge to society, necessitating innovative approaches to understanding its dynamics beyond just merely detecting it. This study introduces a theory-driven framework that integrates advanced natural language processing (NLP) with deep learning, utilizing the target-based emotion–stance analysis (TESA) approach to analyze emotion and stance dynamics within IDO content. Complementing TESA, interactive graph generation (IGG) is applied for scalable and interpretable qualitative analyses. Employing a mixed-methods approach, the study leverages TESA for target-centric emotion and stance analysis, evaluating target-based classifiers on both human-annotated and synthetic datasets. Additionally, the study explores synthetic data generation using generative AI to enrich the analysis, applying IGG to map complex data interactions. The study also found that integrating synthetic data developed from human annotations enhanced model performance, particularly for emotion classification tasks. Results demonstrate that IDO narratives significantly differ from non-IDO narratives, frequently leveraging negative emotions such as anger and disgust to manipulate public perception. TESA proved effective in capturing these nuanced variations, while IGG facilitated the triangulation of such findings via the scalable interpretation of emotional narratives, revealing that IDO content often amplifies polarizing and antagonistic perspectives. By combining TESA and IGG, this research emphasizes the importance of using NLP to extract and examine the emotional and stance nuances toward targets of interest within IDO context. This approach not only deepens theoretical insights into IDO’s persuasive mechanisms but also supports the development of practical tools for analyzing and managing the influence of IDO on public discourse.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"5 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143901304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring Gender Disparities in Experiences of Being Hacked Using Twitter Data: A Focus on the Third-Level Digital Divide 利用Twitter数据探索被黑客攻击经历中的性别差异：关注第三级数字鸿沟

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-04-29 DOI: 10.1177/08944393251334974

Ern Chern Khor, Moon Choi

{"title":"Exploring Gender Disparities in Experiences of Being Hacked Using Twitter Data: A Focus on the Third-Level Digital Divide","authors":"Ern Chern Khor, Moon Choi","doi":"10.1177/08944393251334974","DOIUrl":"https://doi.org/10.1177/08944393251334974","url":null,"abstract":"Despite millions of hacked accounts fueling cybercrime, research on the hacking experience, particularly sociodemographic aspects, remains sparse. This study examines the experience of being hacked with a focus on gender disparities from the perspective of the third-level digital divide—socially constructed gaps of digital use outcomes even among users with similar digital access and skills. Analyzing 13,731 Twitter mentions of accounts being hacked, using topic modeling and classifying the gender of 12,586 users, we showed that women reported more experiences of being hacked across all types of online services except gaming. Women were more likely to experience negative consequences of being hacked, including reputational harm, money loss, and having personalized content modified. Gender differences were also found in coping strategies. Men were more likely to use active strategies like warning others, rebuilding accounts, and deducing hackers’ origins, while women were more likely to seek help from others to recover or report experiencing hacked accounts. The findings of this study imply the need for further research into the gendered experiences of being hacked from the third-level digital divide perspective, alongside the development of interventions to mitigate harm and empower users with diverse needs to cope with being hacked.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"95 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143889529","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Vox Populi, Vox AI? Using Large Language Models to Estimate German Vote Choice 大众之声，人工智能之声？使用大型语言模型估计德国人的投票选择

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-04-26 DOI: 10.1177/08944393251337014

Leah von der Heyde, Anna-Carolina Haensch, Alexander Wenz

{"title":"Vox Populi, Vox AI? Using Large Language Models to Estimate German Vote Choice","authors":"Leah von der Heyde, Anna-Carolina Haensch, Alexander Wenz","doi":"10.1177/08944393251337014","DOIUrl":"https://doi.org/10.1177/08944393251337014","url":null,"abstract":"“Synthetic samples” generated by large language models (LLMs) have been argued to complement or replace traditional surveys, assuming their training data is grounded in human-generated data that potentially reflects attitudes and behaviors prevalent in the population. Initial US-based studies that have prompted LLMs to mimic survey respondents found that the responses match survey data. However, the relationship between the respective target population and LLM training data might affect the generalizability of such findings. In this paper, we critically evaluate the use of LLMs for public opinion research in a different context, by investigating whether LLMs can estimate vote choice in Germany. We generate a synthetic sample matching the 2017 German Longitudinal Election Study respondents and ask the LLM GPT-3.5 to predict each respondent’s vote choice. Comparing these predictions to the survey-based estimates on the aggregate and subgroup levels, we find that GPT-3.5 exhibits a bias towards the Green and Left parties. While the LLM predictions capture the tendencies of “typical” voters, they miss more complex factors of vote choice. By examining the LLM-based prediction of voting behavior in a non-English speaking context, our study contributes to research on the extent to which LLMs can be leveraged for studying public opinion. The findings point to disparities in opinion representation in LLMs and underscore the limitations in applying them for public opinion estimation.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"26 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143878112","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Problematic use of short-video apps among elderly adults: An extension of the TAM 老年人短视频应用的问题使用：TAM的延伸

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-04-25 DOI: 10.1177/08944393251338400

Lingnuo Wang, Guicheng Shi, Jon D. Elhai, Song Zhou, Yiqing Zeng, Lei Zheng

{"title":"Problematic use of short-video apps among elderly adults: An extension of the TAM","authors":"Lingnuo Wang, Guicheng Shi, Jon D. Elhai, Song Zhou, Yiqing Zeng, Lei Zheng","doi":"10.1177/08944393251338400","DOIUrl":"https://doi.org/10.1177/08944393251338400","url":null,"abstract":"Short-form videos have become a dominant form of social media globally. While short-video apps are popular among adolescents, their ease-of-use has also attracted a growing number of elderly users. However, this accessibility can lead to problematic use, resulting in physical and mental health issues for this demographic. Therefore, our research employed the technology acceptance model (TAM) to understand the problematic use of short-video apps (PUSVA) among elderly adults. 281 elderly adults completed a three-wave survey with a 1-month interval between waves. Results showed that both perceived utilitarian-usefulness and perceived hedonic-usefulness mediated the relationship between perceived ease-of-use and PUSVA, suggesting a double-edged sword effect of ease-to-use short-video apps. Moreover, perceived susceptibility moderated the relationship between perceived ease-of-use and perceived utilitarian-usefulness, but not between perceived ease-of-use and perceived hedonic-usefulness, suggesting a moderated mediation effect of perceived susceptibility on PUSVA. Specifically, elderly adults with low perceived susceptibility tended to report higher perceived utilitarian-usefulness for easy-to-use applications, while no relationship between perceived ease-of-use and perceived utilitarian-usefulness was observed among those with high perceived susceptibility. Our findings highlight the double-edged sword effect of user-friendly short-video apps and offer valuable insights for developing interventions to mitigate problematic use among elderly adults.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"9 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143875878","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Social Media Made Me Do It: Perceptions of Social Media Influence, Risky Behaviors, and Mental Health Among Adolescents 社交媒体让我这么做：青少年对社交媒体影响、危险行为和心理健康的看法

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-04-25 DOI: 10.1177/08944393251337016

Robert S. Weisskirch

{"title":"Social Media Made Me Do It: Perceptions of Social Media Influence, Risky Behaviors, and Mental Health Among Adolescents","authors":"Robert S. Weisskirch","doi":"10.1177/08944393251337016","DOIUrl":"https://doi.org/10.1177/08944393251337016","url":null,"abstract":"Adolescents may perceive that social media exert influence on their beliefs, attitudes, and behaviors. Past research has found that frequent social media use and fear of missing out have related to risk behavior and poor mental health outcomes. Little research has been conducted on the perception of influence of social media by adolescents on mental health outcomes and risky behavior engagement. In this study, 304 adolescents (female = 210 and male = 94) completed an online questionnaire about their use of social media, perceptions of social media influence, fear of missing out, engagement in risky behavior, and depressive and anxiety symptoms. Age, perceptions of social media influence, and fear of missing out were significant predictors of engaging in risky behaviors. Age, being female, perceptions of social media influence, and fear of missing out predicted anxiety symptoms. Being female, perceptions of social media influence, and fear of missing out predicted depressive symptoms. For adolescents, the influence of social media on mental health outcomes and risky behaviors may be based on their perception of influence of social media and fear of missing out rather than just frequency of use.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"4 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143875823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Online and Social Media Political Participation: Political Discussion Network Ties and Differential Social Media Platform Effects Over Time 网络与社交媒体政治参与：政治讨论、网络联系与不同时间的社交媒体平台效应

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-04-19 DOI: 10.1177/08944393251332640

Timilehin Durotoye, Manuel Goyanes, Rosa Berganza, Homero Gil de Zúñiga

{"title":"Online and Social Media Political Participation: Political Discussion Network Ties and Differential Social Media Platform Effects Over Time","authors":"Timilehin Durotoye, Manuel Goyanes, Rosa Berganza, Homero Gil de Zúñiga","doi":"10.1177/08944393251332640","DOIUrl":"https://doi.org/10.1177/08944393251332640","url":null,"abstract":"Prior research has largely documented the overall mobilizing effects of social media news consumption and political discussion linked to citizens’ political participatory behaviors. However, limited empirical research has considered the informational and communicative effects to be contingent upon different social media platforms. Therefore, this study advances distinct theoretical affordances and effects of social media news use on online (by using online versions of legacy media outlets, blogs, and news apps) and social media political participation. Taking advantage of US comparative panel data, ordinary least squares (OLS) causal autoregressive regressions and panel autoregressive structural equation model tests cast a much-needed light on the diverse effects of Facebook, X, Snapchat, WhatsApp, Instagram, YouTube, and Reddit use for news over both political discussions with weak and strong ties, and political participation online and in social media. Moreover, results from two-step algorithmic cluster analysis clarify how these social media platforms generate different information and political behavior clusters of citizens, which also provide a comparative view of how social media platforms differently contribute to people’s public and political life in US democracy.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"37 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143849603","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Minority-Owned, Claimed Status, and Profile Attributes of Businesses on Google Maps: COVID-19 Pandemic Survival 谷歌地图上少数人拥有、声称的状态和企业概况属性：COVID-19大流行生存

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-04-14 DOI: 10.1177/08944393251333365

Gi Woong Yun, Sung-Yeon Park

引用次数: 0

Automated Detection of Media Bias Using Artificial Intelligence and Natural Language Processing: A Systematic Review 使用人工智能和自然语言处理的媒体偏见自动检测：系统综述

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-04-04 DOI: 10.1177/08944393251331510

Mar Castillo-Campos, David Becerra-Alonso, Hajo G. Boomgaarden

{"title":"Automated Detection of Media Bias Using Artificial Intelligence and Natural Language Processing: A Systematic Review","authors":"Mar Castillo-Campos, David Becerra-Alonso, Hajo G. Boomgaarden","doi":"10.1177/08944393251331510","DOIUrl":"https://doi.org/10.1177/08944393251331510","url":null,"abstract":"Media bias has long been a subject of scholarly interest due to its potential to shape public perceptions and behaviors. This systematic review leverages advances in natural language processing (NLP) to explore automated methods to detect media bias, addressing five core questions: it examines the definitions and operationalization of media bias, explores the NLP tasks addressed for its detection, the technologies used, and their respective outcomes and applied findings. This review also examines the practical applications of these methodologies and assesses the patterns, implications, and limitations associated with using artificial intelligence for media bias detection. Analyzing peer-reviewed articles from 2019 to 2023, the review initially identified 519 articles, which ultimately included 28 relevant ones. Significant heterogeneity is observed in bias definitions, affecting the analysis and detection approaches. The review highlights the predominant use of some methods and identifies challenges such as inconsistencies in problem definition, outcome measurement, and comparative method evaluation. Regardless of the conceptualizations of bias and the methods used, studies consistently identify bias in media outlets. Thus, studying media bias remains necessary for raising awareness and detection, and NLP methods are significant allies in this endeavor. This research aims to consolidate the foundations of recent advances in NLP for bias detection, encouraging researchers to focus on developing transparent, task-specific tools and work toward a consensus on a technical definition of bias and standardized metrics for its evaluation.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"38 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143782658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Electoral Forecasting in Volatile Party System Settings: Assessing and Improving Pre-Election Poll Predictions in Italy 多变政党制度背景下的选举预测：评估和改进意大利的选前民调预测

IF 4.1 2区社会学

Social Science Computer Review Pub Date : 2025-03-24 DOI: 10.1177/08944393251328309

Kenneth Bunker

{"title":"Electoral Forecasting in Volatile Party System Settings: Assessing and Improving Pre-Election Poll Predictions in Italy","authors":"Kenneth Bunker","doi":"10.1177/08944393251328309","DOIUrl":"https://doi.org/10.1177/08944393251328309","url":null,"abstract":"This study examines electoral forecasting in volatile party systems, focusing on factors contributing to deviations between poll predictions and actual election outcomes. Using Italy as a case study, it identifies biases in polling data and proposes a method to enhance estimator accuracy in a context of stable institutions and volatile electoral dynamics. Data from three Italian general elections are analyzed to evaluate discrepancies between pre-electoral polls and results, assessing key factors such as timing of data collection, survey methodology, sample size, and party system fragmentation. Employing a Bayesian inference process via a Markov chain Monte Carlo (MCMC) adaptive Metropolis-Hastings (MH) algorithm, the study demonstrates that pre-electoral estimates can be significantly improved using the Two-Stage Model (TSM). By consistently outperforming traditional poll predictions, the TSM offers a robust framework for addressing polling biases. These findings advance political forecasting by improving accuracy in both consolidated democracies and volatile electoral contexts, while emphasizing the need for future research on dynamic polling methods and fundamentals-based models.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"183 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2025-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143695280","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0