{"title":"Can Human Reading Validate a Topic Model?","authors":"Bolun Zhang, Yimang Zhou, Dai Li","doi":"10.1177/00811750241265336","DOIUrl":"https://doi.org/10.1177/00811750241265336","url":null,"abstract":"Validation is at the heart of methodological discussions about topic modeling. The authors argue that validation based on human reading hinges on distinctive words and readers’ labeling of a topic, and it overlooks the probability of conflicting results from semantically similar models, such as regressions or other methods. This runs counter to the presumption that topic modeling can reveal features of documents that have some measurable association with social aspects outside the text. The authors develop a similar topic identifying procedure to verify that semantically similar solutions yield similar results in further analysis. The authors argue that future validations of topic modeling must consider such procedures.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":"42 1","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141774347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Moeen Mostafavi, Michael D. Porter, Dawn T. Robinson
{"title":"Contextual Embeddings in Sociological Research: Expanding the Analysis of Sentiment and Social Dynamics","authors":"Moeen Mostafavi, Michael D. Porter, Dawn T. Robinson","doi":"10.1177/00811750241260729","DOIUrl":"https://doi.org/10.1177/00811750241260729","url":null,"abstract":"The authors introduce BERTNN (Bidirectional Encoder Representations from Transformers Neural Network), a novel methodology designed to expand affective lexicons, a critical component in sociological research. BERTNN estimates the affective meanings and their distribution for new concepts, bypassing the need for extensive surveys by leveraging their contextual usage in language. The cornerstone of BERTNN is the use of nuanced word embeddings from Bidirectional Encoder Representations from Transformers. BERTNN uniquely encodes words within the framework of synthesized social event sentences, preserving their meaning across actor-behavior-object positions. The model is fine-tuned on the basis of the implied sentiment changes, providing a more refined estimation of affective meanings. BERTNN outperforms previous approaches, setting a new standard in deriving multidimensional affective meanings for novel concepts. It efficiently replicates sentiment ratings that traditionally require extensive survey hours, demonstrating the power of automated modeling in sociological research. The expanded affective lexicons that can be produced with BERTNN cater to shifting cultural meanings and diverse subgroups, demonstrating the potential of computational linguistics to enrich the measurement tools in sociological research. This article underscores the novelty and significance of BERTNN in the broader context of sociological methodology.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":"40 1","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141774322","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using Relative Distribution Methods to Study Economic Polarization across Categories and Contexts","authors":"Siwei Cheng, Andrew Levine, Ananda Martin-Caughey","doi":"10.1177/00811750241260731","DOIUrl":"https://doi.org/10.1177/00811750241260731","url":null,"abstract":"In addition to overall dispersion, the distributional shape of economic status has attracted growing attention in the inequality literature. Economic polarization is a specific form of distributional change, characterized by a shrinking middle of the distribution and a growing top and bottom, with potentially important and unique social consequences. Building on relative distribution methods and drawing from the literature on job polarization, the authors develop an approach for analyzing economic polarization at the individual level. The method has three useful features. First, it offers intuitive and flexible measurement of economic polarization both between and within categories. Second, it helps disentangle two potential sources of economic polarization: compositional change, which involves changes to the allocation of workers across categories, and relative economic status change, which involves changes to the allocation of economic rewards between individuals. Third, it enables researchers to uncover and examine potential heterogeneity in economic polarization, for example, across occupations, geographic units, demographic and educational groups, and firms. The authors demonstrate the utility of this approach through two empirical applications: (1) an analysis of trends in wage polarization between and within occupations and (2) an examination of geographic variation in income polarization.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":"304 1","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141774327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Question-Order Effect in the Study of Satisfaction with Democracy: Lessons from Three Split-Ballot Experiments","authors":"Zsófia Papp, Pál Susánszky, Andrea Szabó","doi":"10.1177/00811750241254363","DOIUrl":"https://doi.org/10.1177/00811750241254363","url":null,"abstract":"This study examines question-order effects in measuring satisfaction with democracy (SWD). Particularly, the authors are interested in whether the relative position of the question regarding satisfaction with the state of the economy (SWE) in the questionnaire affects responses to the SWD item. The authors conducted three independent split-ballot experiments in Hungary between March 2021 and May 2022. They report a significant and substantial negative priming effect that possibly leads to a systematic underestimation of SWD. Importantly, the authors find no question-order effect in the measurement of SWE. The analysis further reveals a contrast effect: when the SWD question is primed, the difference between SWE and SWD means increases. The authors’ final recommendation is that researchers either put the SWD question before the SWE item to avoid question-order bias or randomize question order. These findings should assist future data collection efforts (comparative or single-country studies) in developing and integrating a battery of satisfaction items into questionnaires and help users assess data quality.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":"68 1","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-05-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141171538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jessica P. Kunke, Ian Laga, Xiaoyue Niu, Tyler H. McCormick
{"title":"Comparing the Robustness of Simple Network Scale-Up Method Estimators","authors":"Jessica P. Kunke, Ian Laga, Xiaoyue Niu, Tyler H. McCormick","doi":"10.1177/00811750241242791","DOIUrl":"https://doi.org/10.1177/00811750241242791","url":null,"abstract":"The network scale-up method (NSUM) is a cost-effective approach to estimating the size or prevalence of a group of people that is hard to reach through a standard survey. The basic NSUM involves two steps: estimating respondents’ degrees and estimating the prevalence of the hard-to-reach population of interest using respondents’ estimated degrees and the number of people they report knowing in the hard-to-reach group. Each of these two steps involves taking either an average of ratios or a ratio of averages. Using the ratio of averages for each step has so far been the most common approach. However, the authors present theoretical arguments that using the average of ratios at the second, prevalence-estimation step often has lower mean squared error when the random mixing assumption is violated, which seems likely in practice; this estimator was proposed early in NSUM development but has largely been unexplored and unused. Simulation results using an example network data set also support these findings. On the basis of this theoretical and empirical evidence, the authors suggest that future surveys that use a simple estimator may want to use this mixed estimator, and estimation methods based on this estimator may produce new improvements.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":"33 1","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140588665","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Multivariate Multinomial Logit Models with Associations among Dependent Variables","authors":"Kazuo Yamaguchi, Jesse Zhou","doi":"10.1177/00811750241239049","DOIUrl":"https://doi.org/10.1177/00811750241239049","url":null,"abstract":"The authors introduce a new group of multinomial logit models with special contrasts to identify covariate effects on multiple categorical dependent variables that are strongly associated with each other. The authors first develop the method for a case with two dependent variables and then extend the method to a case with three dependent variables. The model can account for both nominal and ordinal scales of categorical dependent variables. The authors formulate the covariate effects to represent unique effects on each dependent variable so that they become independent across different dependent variables. The application focuses on the multiplicity of occupational attainments by analyzing how gender, race, educational attainment, and parental occupation characteristics affect three distinct but nonindependent dimensions of occupations: socioeconomic status, social skill level, and math and science skill levels.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":"40 1","pages":""},"PeriodicalIF":3.0,"publicationDate":"2024-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140588551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Loring J. Thomas, Peng Huang, Xiaoshuang Iris Luo, John R. Hipp, Carter T. Butts
{"title":"Marginal-Preserving Imputation of Three-Way Array Data in Nested Structures, with Application to Small Areal Units","authors":"Loring J. Thomas, Peng Huang, Xiaoshuang Iris Luo, John R. Hipp, Carter T. Butts","doi":"10.1177/00811750231203218","DOIUrl":"https://doi.org/10.1177/00811750231203218","url":null,"abstract":"Geospatial population data are typically organized into nested hierarchies of areal units, in which each unit is a union of units at the next lower level. There is increasing interest in analyses at fine geographic detail, but these lowest rungs of the areal unit hierarchy are often incompletely tabulated because of cost, privacy, or other considerations. Here, the authors introduce a novel algorithm to impute crosstabs of up to three dimensions (e.g., race, ethnicity, and gender) from marginal data combined with data at higher levels of aggregation. This method exactly preserves the observed fine-grained marginals, while approximating higher-order correlations observed in more complete higher level data. The authors show how this approach can be used with U.S. census data via a case study involving differences in exposure to crime across demographic groups, showing that the imputation process introduces very little error into downstream analysis, while depicting social process at the more fine-grained level.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":"91 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135341765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Micro Effects on Macro Structure in Social Networks","authors":"Scott W. Duxbury","doi":"10.1177/00811750231209040","DOIUrl":"https://doi.org/10.1177/00811750231209040","url":null,"abstract":"How do individuals’ network selection decisions create unique network structures? Despite broad sociological interest in the micro-level social interactions that create macro-level network structure, few methods are available to statistically evaluate micro-macro relationships in social networks. This study introduces a general methodological framework for testing the effect of (micro) network selection processes, such as homophily, reciprocity, or preferential attachment, on unique (macro) network structures, such as segregation, clustering, or brokerage. The approach uses estimates from a statistical network model to decompose the contributions of each parameter to a node, subgraph, or global network statistic specified by the researcher. A flexible parametric algorithm is introduced to estimate variances, confidence intervals, and p values. Prior micro-macro network methods can be regarded as special cases of the general framework. Extensions to hypothetical network interventions, joint parameter tests, and longitudinal and multilevel network data are discussed. An example is provided analyzing the micro foundations of political segregation in a crime policy collaboration network.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":"29 36","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135391263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Networked Participants, Networked Meanings: Using Networks to Visualize Ethnographic Data","authors":"Kenneth R. Hanson, Nicholas Theis","doi":"10.1177/00811750231195338","DOIUrl":"https://doi.org/10.1177/00811750231195338","url":null,"abstract":"Researchers can use data visualization techniques to explore, analyze, and present data in new ways. Although quantitative data are visualized most often, recent innovations have brought attention to the potential benefits of visualizing qualitative data. In this article, the authors demonstrate one way researchers can use networks to analyze and present ethnographic interview data. The authors suggest that because many respondents know one another in ethnographic research, networks are a useful tool for analyzing the implications of respondents’ familiarity with one another. Moreover, respondents often share familiar cultural references that can be visualized. The authors show how visualizing respondents’ ties in conjunction with their shared cultural references sheds light on the different systems of meaning that respondents within a field site use to make sense of the social phenomena under investigation.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":"1 1","pages":""},"PeriodicalIF":3.0,"publicationDate":"2023-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41505157","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Trend Analysis with Pooled Data from Different Survey Series: The Latent Attitude Method","authors":"Donghui Wang, Yueqi Xie, Junming Huang","doi":"10.1177/00811750231193641","DOIUrl":"https://doi.org/10.1177/00811750231193641","url":null,"abstract":"The use of pooled data from different repeated survey series to study long-term trends is handicapped by a measurement difficulty: different survey series often use different scales to measure the same attitude and thus generate scale-incomparable data. In this article, the authors propose the latent attitude method (LAM) to address this scale-incomparability problem, on the basis of the assumption that attitudes measured by ordinal categories reflect a latent attitude with cut points. The method extends the latent variable method in the case of a single survey series to the case of multiple survey series and leverages overlapping years for identification. The authors first assess the validity of the method with simulated data. The results show that the method yields accurate estimates of mean attitudes and cut point values. The authors then apply the method to an empirical study of Americans’ attitudes toward China from 1974 to 2019.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":" ","pages":""},"PeriodicalIF":3.0,"publicationDate":"2023-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45016206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}