Sociological Methods & Research最新文献

Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning 用于文本分类的大型语言模型：从零学习到指令调整

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-04-24 DOI: 10.1177/00491241251325243

Youngjin Chae, Thomas Davidson

{"title":"Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning","authors":"Youngjin Chae, Thomas Davidson","doi":"10.1177/00491241251325243","DOIUrl":"https://doi.org/10.1177/00491241251325243","url":null,"abstract":"Large language models (LLMs) have tremendous potential for social science research as they are trained on vast amounts of text and can generalize to many tasks. We explore the use of LLMs for supervised text classification, specifically the application to stance detection, which involves detecting attitudes and opinions in texts. We examine the performance of these models across different architectures, training regimes, and task specifications. We compare 10 models ranging in size from tens of millions to hundreds of billions of parameters and test four distinct training regimes: Prompt-based zero-shot learning and few-shot learning, fine-tuning, and instruction-tuning, which combines prompting and fine-tuning. The largest, most powerful models generally offer the best predictive performance even with little or no training examples, but fine-tuning smaller models is a competitive solution due to their relatively high accuracy and low cost. Instruction-tuning the latest generative LLMs expands the scope of text classification, enabling applications to more complex tasks than previously feasible. We offer practical recommendations on the use of LLMs for text classification in sociological research and discuss their limitations and challenges. Ultimately, LLMs can make text classification and other text analysis methods more accurate, accessible, and adaptable, opening new possibilities for computational social science.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"72 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143866960","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Conceptualizing Job and Employment Concepts for Earnings Inequality Estimands With Linked Employer-Employee Data 1 概念化工作和就业概念与关联雇主-雇员数据的收入不平等估计1

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-04-24 DOI: 10.1177/00491241251334124

Donald Tomaskovic-Devey, Chen-Shuo Hong

引用次数: 0

The Target Study: A Conceptual Model and Framework for Measuring Disparity 目标研究：衡量差异的概念模型和框架

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-04-22 DOI: 10.1177/00491241251314037

John W. Jackson, Yea-Jen Hsu, Raquel C. Greer, Romsai T. Boonyasai, Chanelle J. Howe

引用次数: 0

Networks Beyond Categories: A Computational Approach to Examining Gender Homophily 超越类别的网络：研究性别同源性的计算方法

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-04-22 DOI: 10.1177/00491241251321152

Chen-Shuo Hong

{"title":"Networks Beyond Categories: A Computational Approach to Examining Gender Homophily","authors":"Chen-Shuo Hong","doi":"10.1177/00491241251321152","DOIUrl":"https://doi.org/10.1177/00491241251321152","url":null,"abstract":"Social networks literature has explored homophily, the tendency to associate with similar others, as a critical boundary-making process contributing to segregated networks along the lines of identities. Yet, social network research generally conceptualizes identities as sociodemographic categories and seldom considers the inherently continuous and heterogeneous nature of differences. Drawing upon the infracategorical model of inequality, this study demonstrates that a computational approach – combining machine learning and exponential random graph models (ERGMs) – can capture the role of categorical conformity in network structures. Through a case study of gender segregation in friendships, this study presents a workflow for developing a machine-learning-based gender conformity measure and applying it to guide the social network analysis of cultural matching. Results show that adolescents with similar gender conformity are more likely to form friendships, net of homophily based on categorical gender and other controls, and homophily by gender conformity mediates homophily by categorical gender. The study concludes by discussing the limitations of this computational approach and its unique strengths in enhancing theories on categories, boundaries, and stratification.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"32 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143862884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Mixed Subjects Design: Treating Large Language Models as Potentially Informative Observations 混合主题设计：将大型语言模型视为潜在的信息观察

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-04-22 DOI: 10.1177/00491241251326865

David Broska, Michael Howes, Austin van Loon

{"title":"The Mixed Subjects Design: Treating Large Language Models as Potentially Informative Observations","authors":"David Broska, Michael Howes, Austin van Loon","doi":"10.1177/00491241251326865","DOIUrl":"https://doi.org/10.1177/00491241251326865","url":null,"abstract":"Large language models (LLMs) provide cost-effective but possibly inaccurate predictions of human behavior. Despite growing evidence that predicted and observed behavior are often not interchangeable , there is limited guidance on using LLMs to obtain valid estimates of causal effects and other parameters. We argue that LLM predictions should be treated as potentially informative observations, while human subjects serve as a gold standard in a mixed subjects design . This paradigm preserves validity and offers more precise estimates at a lower cost than experiments relying exclusively on human subjects. We demonstrate—and extend—prediction-powered inference (PPI), a method that combines predictions and observations. We define the PPI correlation as a measure of interchangeability and derive the effective sample size for PPI. We also introduce a power analysis to optimally choose between informative but costly human subjects and less informative but cheap predictions of human behavior. Mixed subjects designs could enhance scientific productivity and reduce inequality in access to costly evidence.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"4 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143862886","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Social Mobility as Causal Intervention 社会流动作为因果干预

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-04-21 DOI: 10.1177/00491241251320963

Lai Wei, Yu Xie

{"title":"Social Mobility as Causal Intervention","authors":"Lai Wei, Yu Xie","doi":"10.1177/00491241251320963","DOIUrl":"https://doi.org/10.1177/00491241251320963","url":null,"abstract":"The study of mobility effects is an important subject of study in sociology. Empirical investigations of individual mobility effects, however, have been hindered by one fundamental limitation, the unidentifiability of mobility effects when origin and destination are held constant. Given this fundamental limitation, we propose to reconceptualize mobility effects from the micro- to macro-level. Instead of micro-level mobility effects, the primary focus of the past literature, we ask alternative research questions about macro-level mobility effects: What happens to the population distribution of an outcome if we manipulate the mobility regime, that is, if we alter the observed association between social origin and social destination? We relate individual-level mobility experience to macro-level mobility effects under special interventions. The proposed method bridges the macro and micro agendas in social stratification research, and has wider applications in social stratification beyond the study of mobility effects. We illustrate the method with two analyses that evaluate the impact of social mobility on average fertility and income inequality in the United States. We provide an open-source software, the R package socmob , that implements the method.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"1 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143857717","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Machine Bias. How Do Generative Language Models Answer Opinion Polls? 机器的偏见。生成语言模型如何回答民意调查？

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-04-21 DOI: 10.1177/00491241251330582

Julien Boelaert, Samuel Coavoux, Étienne Ollion, Ivaylo Petev, Patrick Präg

引用次数: 0

Correcting the Measurement Errors of AI-Assisted Labeling in Image Analysis Using Design-Based Supervised Learning 基于设计的监督学习修正图像分析中人工智能辅助标注的测量误差

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-04-21 DOI: 10.1177/00491241251333372

Alessandra Rister Portinari Maranca, Jihoon Chung, Musashi Hinck, Adam D. Wolsky, Naoki Egami, Brandon M. Stewart

{"title":"Correcting the Measurement Errors of AI-Assisted Labeling in Image Analysis Using Design-Based Supervised Learning","authors":"Alessandra Rister Portinari Maranca, Jihoon Chung, Musashi Hinck, Adam D. Wolsky, Naoki Egami, Brandon M. Stewart","doi":"10.1177/00491241251333372","DOIUrl":"https://doi.org/10.1177/00491241251333372","url":null,"abstract":"Generative artificial intelligence (AI) has shown incredible leaps in performance across data of a variety of modalities including texts, images, audio, and videos. This affords social scientists the ability to annotate variables of interest from unstructured media. While rapidly improving, these methods are far from perfect and, as we show, even ignoring the small amounts of error in high accuracy systems can lead to substantial bias and invalid confidence intervals in downstream analysis. We review how using design-based supervised learning (DSL) guarantees asymptotic unbiasedness and proper confidence interval coverage by making use of a small number of expert annotations. While originally developed for use with large language models in text, we present a series of applications in the context of image analysis, including an investigation of visual predictors of the perceived level of violence in protest images, an analysis of the images shared in the Black Lives Matter movement on Twitter, and a study of U.S. outlets reporting of immigrant caravans. These applications are representative of the type of analysis performed in the visual social science landscape today, and our analyses will exemplify how DSL helps us attain statistical guarantees while using automated methods to reduce human labor.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"3 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143857722","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Insight-Inference Loop: Efficient Text Classification via Natural Language Inference and Threshold-Tuning 洞察-推理循环：基于自然语言推理和阈值调优的高效文本分类

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-04-19 DOI: 10.1177/00491241251326819

Sandrine Chausson, Marion Fourcade, David J. Harding, Björn Ross, Grégory Renard

{"title":"The Insight-Inference Loop: Efficient Text Classification via Natural Language Inference and Threshold-Tuning","authors":"Sandrine Chausson, Marion Fourcade, David J. Harding, Björn Ross, Grégory Renard","doi":"10.1177/00491241251326819","DOIUrl":"https://doi.org/10.1177/00491241251326819","url":null,"abstract":"Modern computational text classification methods have brought social scientists tantalizingly close to the goal of unlocking vast insights buried in text data—from centuries of historical documents to streams of social media posts. Yet three barriers still stand in the way: the tedious labor of manual text annotation, the technical complexity that keeps these tools out of reach for many researchers, and, perhaps most critically, the challenge of bridging the gap between sophisticated algorithms and the deep theoretical understanding social scientists have already developed about human interactions, social structures, and institutions. To counter these limitations, we propose an approach to large-scale text analysis that requires substantially less human-labeled data, and no machine learning expertise, and efficiently integrates the social scientist into critical steps in the workflow. This approach, which allows the detection of statements in text, relies on large language models pre-trained for natural language inference, and a “few-shot” threshold-tuning algorithm rooted in active learning principles. We describe and showcase our approach by analyzing tweets collected during the 2020 U.S. presidential election campaign, and benchmark it against various computational approaches across three datasets.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"1 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2025-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143851025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Locating Cultural Holes Brokers in Diffusion Dynamics Across Bright Symbolic Boundaries 在跨越明亮符号边界的扩散动力学中定位文化漏洞经纪人

IF 6.3 2区社会学

Sociological Methods & Research Pub Date : 2025-03-19 DOI: 10.1177/00491241251322517

Diego F. Leal

引用次数: 0