{"title":"Agreement Helps, Disagreement Matters: Scaling Framing Function Annotation With Large Language Models","authors":"Emily Robinson, Ranadheer Malla, Chico Q. Camargo","doi":"10.1177/08944393261501889","DOIUrl":"https://doi.org/10.1177/08944393261501889","url":null,"abstract":"Framing research has long relied on human annotation, but many aspects of framing – especially framing functions such as defining problems, identifying causes, making moral evaluations, or proposing solutions are difficult to label consistently and at scale. We demonstrate that large language models can reliably identify these framing functions, achieving levels of agreement comparable to those of human annotators. We evaluate three state-of-the-art models (GPT-5, Claude 3.7, and DeepSeek) alongside three trained human coders on a new dataset of social media posts about electric-vehicles, a highly topical and contested issue that holds many interpretations. This setting provides a strong test of both agreement and meaningful disagreement in framing-function annotation. Across models and humans, we find not only substantial convergence but also patterns of valid disagreement – differences that reflect legitimate alternative interpretations rather than annotation error. Recognising and preserving these cases enhances construct validity and provides a more realistic portrayal of how framing functions operate in contested debates. Our results demonstrate that large language models can scale framing function analysis while capturing the variability inherent in human interpretation, enabling forms of communication research that were previously impractical with human-only coding.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"16 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149287790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Cost of Capturing: How Smartphone Photo-Taking Alters Memory Performance and Cognitive Self-Esteem: The Role of Smartphone Self-Expansion","authors":"Tianke Che, Wenxin Xu, Xiaochun Xie","doi":"10.1177/08944393261502771","DOIUrl":"https://doi.org/10.1177/08944393261502771","url":null,"abstract":"Taking photos to manage memories with smartphones is a routine strategy in the digital era. However, it remains unclear how smartphone photo-taking affects memory performance and self-evaluation. This research examined whether smartphone photo-taking is associated with a pattern consistent with the “Google effect,” and whether these associations vary with self-expansion via smartphone use. Across three experiments ( <jats:italic toggle=\"yes\">N</jats:italic> = 363), participants photographed or observed materials before completing memory tests and measures of cognitive self-esteem. Study 1 tested the basic photo-taking effect; Study 2a examined device ownership as a contextual proxy for smartphone self-expansion; and Study 2b examined trait self-expansion as a continuous moderator. Results showed a photo-taking-impairment effect on memory, alongside higher cognitive self-esteem following photo-taking. Moreover, higher smartphone self-expansion was associated with stronger increases in cognitive self-esteem and greater memory overestimation, whereas its role in objective memory impairment remained inconclusive. Altogether, the findings suggest that smartphone photo-taking can reduce memory performance while increasing cognitive self-esteem.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"50 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149281347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Pathways Out of Cybercrime: Perceived Desistance From Criminal Hacking","authors":"E. Rutger Leukfeldt, Luuk M. J. Bekkers","doi":"10.1177/08944393261501868","DOIUrl":"https://doi.org/10.1177/08944393261501868","url":null,"abstract":"The present study seeks to understand how criminal hackers self-report to desist from the world of cybercrime. In doing so, we address significant gaps in criminological literature, as little is known about pathways out of cybercrime offending. Interviews were conducted with 22 individuals involved in cyber-dependent crime, primarily from the Netherlands. Of these, 15 respondents were justice-involved youth and 7 respondents self-identified as criminal hackers. The interviews offer unique insights into the stages offenders navigate following their involvement in cybercrime. Findings reveal that young hackers may be susceptible to behavioral change through various mechanisms. Desistance from cybercrime is a process that warrants a multidisciplinary perspective, influenced particularly by social bonds, prosocial daily routines, and deterrent experiences. A combination of police intervention and alternative approaches that emphasize ethical application of skills may be a promising route for young hackers in particular, although actual intervention effects remain unverified. We encourage evaluations of current programs for hackers as well as additional quantitative assessments of protective factors among cybercrime offenders.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"3 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149281359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Academic Passporting in Western Versus Non-Western Feminist Scholarship (1936–2024): Bibliometric and Thematic Landscapes","authors":"Muhammad Awais","doi":"10.1177/08944393261493961","DOIUrl":"https://doi.org/10.1177/08944393261493961","url":null,"abstract":"This study examines structural inequalities in feminist knowledge production by analyzing 11,060 Scopus indexed articles published between 1936 and 2024. Using bibliometric indicators, statistical citation analysis, and topic modeling, the study found that Western affiliated scholars dominate both output and recognition, whereas non-Western voices remain numerically and symbolically marginalized. The findings highlight how “Westernization” operates not only through citation advantage but also through thematic centrality: scholarship from Western institutions often sets the agenda in areas such as digital activism and intersectionality, while non-Western contributions, often centered on decolonial critique, grassroots mobilization, and local contexts, receive less global visibility. Mixed teams improve recognition among non-Western scholars but do not close the structural gap, suggesting that collaboration narrows inequalities without redistributing epistemic authority. To explain these dynamics, this study introduces the concept of <jats:italic toggle=\"yes\">academic passporting</jats:italic> , whereby Western institutional affiliations function as strong passports that confer recognition and visibility, whereas non-Western affiliations operate as weak passports, unless stamped through Western collaboration. These findings have implications for diversifying scholarly practices, rethinking theoretical frameworks, and shaping equitable research policies.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"33 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149198906","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Francesco Nicoli,Gregorio Buzzelli,Brian Burgoon,Stefano Sacchi
{"title":"Who Is Afraid of ChatGPT? A Natural Experiment on the Effect of ChatGPT Release on Labour-Market Concerns","authors":"Francesco Nicoli,Gregorio Buzzelli,Brian Burgoon,Stefano Sacchi","doi":"10.1177/08944393261493959","DOIUrl":"https://doi.org/10.1177/08944393261493959","url":null,"abstract":"The public release of ChatGPT on November 30th, 2022 revived the public debate on technological change and, particularly, on its impact on the labour market. Despite the sudden growth in AI salience after the launch of the new chatbot, to our knowledge, no empirical investigation has assessed the impact of this technological breakthrough on citizens’ first concerns. By means of a unique survey fielded between November and December 2022 in five European countries, we use an Unexpected Event During Survey (UEDS) design to causally test the effect of ChatGPT’s launch on individual perception of unemployment risk. We rely on statistical matching techniques, combined with regression analyses to explore heterogeneity in responses between socioeconomic groups. We show that the release of the new chatbot mildly but significantly increased unemployment concerns in advanced industrial democracies, particularly among individuals with low and middling levels of income and education, as well as poorly exposed to AI. By capturing reactions before informational framings consolidate and widespread adoption occurs, this study provides a rare window into citizens’ genuine response to technological change, indicating that its socioeconomic consequences are perceived as immediately salient.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"167 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149151754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Leveraging Structural Information to Reconstruct Hidden Nodes in Propagation Graphs","authors":"Damian Frąszczak","doi":"10.1177/08944393261485322","DOIUrl":"https://doi.org/10.1177/08944393261485322","url":null,"abstract":"While widely used for information exchange, social media networks are increasingly exploited to spread rumors and malicious content. It is essential to reconstruct propagation graphs to hold nodes accountable for such propagation. However, the dynamic and vast nature of social networks makes this task challenging. This study introduces Structural Hidden Node Identification (SHNI), a heuristic method for reconstructing partially observed propagation snapshots by identifying likely hidden participants. SHNI relies on the observed active nodes and structural neighborhood information, without requiring temporal traces, message content, user attributes, or network-specific supervised retraining; its structural parameters may be selected by expert knowledge or calibrated once and then reused across networks. Instead of materializing the entire graph in advance, the method supports on-demand neighborhood expansion from observed active nodes toward structurally relevant candidate nodes. It combines local activation pressure with a structural bridging component inspired by sociological principles of opinion formation, including local social influence, information bubbles, and the role of bridging nodes between groups. SHNI explores candidate hidden nodes through a priority-driven neighborhood expansion strategy, in which the most promising candidates are evaluated first, and further exploration stops when the heuristic evidence becomes insufficient. Experiments across different network structures show that SHNI achieves competitive reconstruction quality while maintaining low data requirements and transparent decision logic. These results indicate that SHNI is especially useful as an interpretable structural baseline when temporal traces, message content, or network-specific supervised training data are unavailable. The method may also be adapted to other propagation phenomena, such as epidemic spread or malware diffusion in computer networks.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"10 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148979553","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Marcello Carammia, Stefano Maria Iacus, Giuseppe Porro
{"title":"Rethinking Scale: The Efficacy of Fine-Tuned Open-Source LLMs in Large-Scale Reproducible Social Science Research","authors":"Marcello Carammia, Stefano Maria Iacus, Giuseppe Porro","doi":"10.1177/08944393261485320","DOIUrl":"https://doi.org/10.1177/08944393261485320","url":null,"abstract":"Large language models (LLMs) are distinguished by their architecture, parameter size, and performance capabilities. Social scientists have increasingly adopted LLMs to automate high-dimensional text classification tasks which are difficult to scale with human coders. Very large, closed-source models often deliver superior performance. However, reliance on large proprietary models raises concerns about transparency, reproducibility, and costs that make them impractical for large-scale research projects. In contrast, open-source models offer distinct advantages: they can be run locally (ensuring data privacy), fine-tuned for specific tasks, shared within the research community, and integrated into reproducible workflows. On the other hand, open-source models often underperform compared to commercial alternatives. In this study, we assess whether small, open-source LLMs – when fine-tuned appropriately – can match or exceed the performance of commercial alternatives such as GPT-4, while remaining feasible for large-scale academic research. To produce the labeled data that fine-tuning requires, we develop a hybrid annotation workflow in which several LLMs propose labels that human reviewers verify; this infrastructure is what makes fine-tuning feasible at scale. We apply this pipeline to three substantive social science classification tasks: coding tweets for human flourishing indicators, classifying European Parliament questions by policy area, and tagging datasets from the Harvard Dataverse. Our results show that fine-tuned models as small as Llama2-7B achieve performance on par with much larger closed models, offering a reproducible, low-cost alternative for computational social science research. We also explore the limits of fine-tuning effectiveness across model sizes and training set volumes. This work demonstrates a scalable and replicable approach to integrating open-source LLMs into empirical social science workflows.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"1 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148979505","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rasoul Norouzi, Bennett Kleinberg, Jeroen Vermunt, Caspar Van Lissa
{"title":"Parsing Causal Relationships in Social Science Publications","authors":"Rasoul Norouzi, Bennett Kleinberg, Jeroen Vermunt, Caspar Van Lissa","doi":"10.1177/08944393261478561","DOIUrl":"https://doi.org/10.1177/08944393261478561","url":null,"abstract":"Understanding the causes of phenomena is a key goal of scientific enquiry. For historic reasons, social scientists are reluctant to explicitly discuss causal assumptions. Nevertheless, causal claims do appear in the literature. Systematically cataloging these causal claims and parsing them into structured cause-effect relationship would allow us to synthesize them and, ultimately, construct theories based on oft-repeated causal claims. However, manually extracting these claims is infeasible due to the scale of the literature and linguistic ambiguity, and existing automated methods often lack domain specificity or fail to extract specific cause-effect pairs. To overcome these limitations, we introduce a BERT-based multitask deep learning model trained on a novel, domain-specific benchmark dataset of 3,014 annotated sentences from social science papers. The model simultaneously (1) identifies sentences containing causal claims, (2) extracts cause and effect spans, and (3) links the associated causal pairs. In our study, the multi-head model outperformed a model with sequential architecture, as well as two generative LLMs (Llama 3 8B and Qwen3 8B), which were prompted to code the sentences with a few examples of the coding schema. Specifically, the multi-head model had the highest macro F1-score across all three subtasks and lower false-positive error propagation compared to the sequential model. Its agreement with human coders on the held-out test set (Krippendorff’s α = 0.81) was of similar magnitude to the interrater agreement among the human coders, computed on an interrater reliability subsample (α = 0.80).","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"16 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148915406","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Wild, Thick, and Wicked: Situated Evidence on AI-In-Use for Decisions About Deploying AI Systems","authors":"Reva Schwartz, Gabriella Waters","doi":"10.1177/08944393261481511","DOIUrl":"https://doi.org/10.1177/08944393261481511","url":null,"abstract":"Organizations are adopting generative AI faster than the evidence base needed to govern it. Existing evaluation tools such as benchmarks, alignment scores, and safety tests were built for model development, not for judging whether systems will create value, introduce friction, or shift risk in specific real-world settings. As a result, there is little systematic evidence about how AI behaves once it is embedded in everyday work. This paper proposes a real-world AI evaluation framework focused on AI-in-use: how people actually appropriate, adapt, and work around AI systems in context, and what consequences follow over time. Instead of treating variability across users, tasks, and settings as noise to be controlled away, the framework treats that variation as the central source of deployment-relevant evidence. It sets out four design principles for producing decision-ready evidence at scale and proposes a shared evaluation architecture combining a structured observation environment, a metrics hub, and reusable consortium models that summarize system behavior across contexts. Rather than replacing traditional benchmarks, this framework adds a sociotechnical evidence layer that connects model capabilities to the organizational and practitioner level outcomes where deployment decisions are actually made.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"5 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148899618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sean Hamilton-Palicki, Isaac Bravo, Clint Claessen
{"title":"Don’t Look up: Evaluating the Tradeoff Between Performance and Sustainability of Text Classification Using Open LLMs","authors":"Sean Hamilton-Palicki, Isaac Bravo, Clint Claessen","doi":"10.1177/08944393261473230","DOIUrl":"https://doi.org/10.1177/08944393261473230","url":null,"abstract":"The increasing adoption of Large Language Models (LLMs) as a text analysis method in social science presents a critical yet under-examined trade-off between model performance and environmental sustainability. This research provides a systematic evaluation comparing the performance, energy consumption, processing time, and CO <jats:sub>2</jats:sub> emissions of various computational text analysis methods (CTAM), including dictionaries, trained classifiers, and self-hosted open LLMs when performing sentiment analysis of parliamentary speeches, classification of open-ended survey responses, and named entity recognition of newspapers. The analysis is limited to self-hosted deployment in local and server environments where per-task energy consumption is directly measurable. Although self-hosted LLMs demonstrate strong performance in sentiment analysis, closely aligning with human judgment, they require significantly more energy and time than non-LLM approaches. For classification and named entity recognition, pretrained task-specific models achieve better F1 scores with a lower carbon footprint, challenging the primacy of larger models. To navigate this trade-off, we propose a CO <jats:sub>2</jats:sub> -Adjusted F1 Score that penalizes emissions while rewarding performance. Applying this metric, we show that smaller, task-specific models may be preferred over larger general-purpose LLMs for efficient text analysis. We highlight the necessity for thoughtful and responsible model selection, promoting a “right-fit” approach for CTAM.","PeriodicalId":49509,"journal":{"name":"Social Science Computer Review","volume":"40 1","pages":""},"PeriodicalIF":4.1,"publicationDate":"2026-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148877280","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}