{"title":"Who Does What to Whom? Making Text Parsers Work for Sociological Inquiry","authors":"Oscar Stuhler","doi":"10.1177/00491241221099551","DOIUrl":"https://doi.org/10.1177/00491241221099551","url":null,"abstract":"Over the past decade, sociologists have become increasingly interested in the formal study of semantic relations within text. Most contemporary studies focus either on mapping concept co-occurrences or on measuring semantic associations via word embeddings. Although conducive to many research goals, these approaches share an important limitation: they abstract away what one can call the event structure of texts, that is, the narrative action that takes place in them. I aim to overcome this limitation by introducing a new framework for extracting semantically rich relations from text that involves three components. First, a semantic grammar structured around textual entities that distinguishes six motif classes: actions of an entity, treatments of an entity, agents acting upon an entity, patients acted upon by an entity, characterizations of an entity, and possessions of an entity; second, a comprehensive set of mapping rules, which make it possible to recover motifs from predictions of dependency parsers; third, an R package that allows researchers to extract motifs from their own texts. The framework is demonstrated in empirical analyses on gendered interaction in novels and constructions of collective identity by U.S. presidential candidates.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"51 1","pages":"1580 - 1633"},"PeriodicalIF":6.3,"publicationDate":"2022-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43636799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Language-Based Method for Assessing Symbolic Boundary Maintenance between Social Groups","authors":"Anjali M. Bhatt, Amir Goldberg, S. Srivastava","doi":"10.1177/00491241221099555","DOIUrl":"https://doi.org/10.1177/00491241221099555","url":null,"abstract":"When the social boundaries between groups are breached, the tendency for people to erect and maintain symbolic boundaries intensifies. Drawing on extant perspectives on boundary maintenance, we distinguish between two strategies that people pursue in maintaining symbolic boundaries: boundary retention—entrenching themselves in pre-existing symbolic distinctions—and boundary reformation—innovating new forms of symbolic distinction. Traditional approaches to measuring symbolic boundaries—interviews, participant-observation, and self-reports are ill-suited to detecting fine-grained variation in boundary maintenance. To overcome this limitation, we use the tools of computational linguistics and machine learning to develop a novel approach to measuring symbolic boundaries based on interactional language use between group members before and after they encounter one another. We construct measures of boundary retention and reformation using random forest classifiers that quantify group differences based on pre- and post-contact linguistic styles. We demonstrate this method's utility by applying it to a corpus of email communications from a mid-sized financial services firm that acquired and integrated two smaller firms. We find that: (a) the persistence of symbolic boundaries can be detected for up to 18 months after a merger; (b) acquired employees exhibit more boundary reformation and less boundary retention than their counterparts from the acquiring firm; and (c) individuals engage in more boundary retention, but not reformation, when their local work environment is more densely populated by ingroup members. We discuss implications of these findings for the study of culture in a wide range of intergroup contexts and for computational approaches to measuring culture.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"51 1","pages":"1681 - 1720"},"PeriodicalIF":6.3,"publicationDate":"2022-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48787423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"BIC extensions for order-constrained model selection.","authors":"J Mulder, A E Raftery","doi":"10.1177/0049124119882459","DOIUrl":"https://doi.org/10.1177/0049124119882459","url":null,"abstract":"<p><p>The Schwarz or Bayesian information criterion (BIC) is one of the most widely used tools for model comparison in social science research. The BIC however is not suitable for evaluating models with order constraints on the parameters of interest. This paper explores two extensions of the BIC for evaluating order constrained models, one where a truncated unit information prior is used under the order-constrained model, and the other where a truncated local unit information prior is used. The first prior is centered around the maximum likelihood estimate and the latter prior is centered around a null value. Several analyses show that the order-constrained BIC based on the local unit information prior works better as an Occam's razor for evaluating order-constrained models and results in lower error probabilities. The methodology based on the local unit information prior is implemented in the R package 'BFpack' which allows researchers to easily apply the method for order-constrained model selection. The usefulness of the methodology is illustrated using data from the European Values Study.</p>","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"51 2","pages":"471-498"},"PeriodicalIF":6.3,"publicationDate":"2022-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/0049124119882459","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40491402","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Meta-Analysis in Sociological Research: Power and Heterogeneity.","authors":"Guangyu Tong, Guang Guo","doi":"10.1177/0049124119882479","DOIUrl":"https://doi.org/10.1177/0049124119882479","url":null,"abstract":"<p><p>Meta-analysis is a statistical method that combines quantitative findings from previous studies. It has been increasingly used to obtain more credible results in a wide range of scientific fields. Combining the results of relevant studies allows researchers to leverage study similarities while modeling potential sources of between-study heterogeneity. This paper provides a review of the core methodologies of meta-analysis that we consider most relevant to sociological research. After developing the foundation of the fixed-effects and random-effects models of meta-analysis, this paper illustrates the utility of the method with regression coefficients reported from two sets of social science studies. We explain the various steps of the process including constructing the meta-sample from primary studies; estimating the fixed- and random-effects models; analyzing the source of heterogeneity across studies; assessing publication bias. We conclude with a discussion of steps that could be taken to strengthen the development of meta-analysis in sociological research, which will eventually increase the credibility of sociological inquiry via a knowledge-cumulative process.</p>","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"51 2","pages":"566-604"},"PeriodicalIF":6.3,"publicationDate":"2022-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/0049124119882479","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40400611","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
B. Meuleman, Tomasz Żółtak, A. Pokropek, E. Davidov, B. Muthén, Daniel L. Oberski, J. Billiet, Peter Schmidt
{"title":"Why Measurement Invariance is Important in Comparative Research. A Response to Welzel et al. (2021)","authors":"B. Meuleman, Tomasz Żółtak, A. Pokropek, E. Davidov, B. Muthén, Daniel L. Oberski, J. Billiet, Peter Schmidt","doi":"10.1177/00491241221091755","DOIUrl":"https://doi.org/10.1177/00491241221091755","url":null,"abstract":"Welzel et al. (2021) claim that non-invariance of instruments is inconclusive and inconsequential in the field for cross-cultural value measurement. In this response, we contend that several key arguments on which Welzel et al. (2021) base their critique of invariance testing are conceptually and statistically incorrect. First, Welzel et al. (2021) claim that value measurement follows a formative rather than reflective logic. Yet they do not provide sufficient theoretical arguments for this conceptualization, nor do they discuss the disadvantages of this approach for validation of instruments. Second, their claim that strong inter-item correlations cannot be retrieved when means are close to the endpoint of scales ignores the existence of factor-analytic approaches for ordered-categorical indicators. Third, Welzel et al. (2021) propose that rather than of relying on invariance tests, comparability can be assessed by studying the connection with theoretically related constructs. However, their proposal ignores that external validation through nomological linkages hinges on the assumption of comparability. By means of two examples, we illustrate that violating the assumptions of measurement invariance can distort conclusions substantially. Following the advice of Welzel et al. (2021) implies discarding a tool that has proven to be very useful for comparativists.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"52 1","pages":"1401 - 1419"},"PeriodicalIF":6.3,"publicationDate":"2022-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46916158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Against the Mainstream: On the Limitations of Non-Invariance Diagnostics: Response to Fischer et al. and Meuleman et al.","authors":"C. Welzel, S. Kruse, Lennart Brunkert","doi":"10.1177/00491241221091754","DOIUrl":"https://doi.org/10.1177/00491241221091754","url":null,"abstract":"Our original 2021 SMR article “Non-Invariance? An Overstated Problem with Misconceived Causes” disputes the conclusiveness of non-invariance diagnostics in diverse cross-cultural settings. Our critique targets the increasingly fashionable use of Multi-Group Confirmatory Factor Analysis (MGCFA), especially in its mainstream version. We document—both by mathematical proof and an empirical illustration—that non-invariance is an arithmetic artifact of group mean disparity on closed-ended scales. Precisely this arti-factualness renders standard non-invariance markers inconclusive of measurement inequivalence under group-mean diversity. Using the Emancipative Values Index (EVI), OA-Section 3 of our original article demonstrates that such artifactual non-invariance is inconsequential for multi-item constructs’ cross-cultural performance in nomological terms, that is, explanatory power and predictive quality. Given these limitations of standard non-invariance diagnostics, we challenge the unquestioned authority of invariance tests as a tool of measurement validation. Our critique provoked two teams of authors to launch counter-critiques. We are grateful to the two comments because they give us a welcome opportunity to restate our position in greater clarity. Before addressing the comments one by one, we reformulate our key propositions more succinctly.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"52 1","pages":"1438 - 1455"},"PeriodicalIF":6.3,"publicationDate":"2022-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41829124","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
L. Vila‐Henninger, C. Dupuy, Virginie Van Ingelgom, M. Caprioli, Ferdinand Teuber, Damien Pennetreau, Margherita Bussi, Cal Le Gall
{"title":"Abductive Coding: Theory Building and Qualitative (Re)Analysis","authors":"L. Vila‐Henninger, C. Dupuy, Virginie Van Ingelgom, M. Caprioli, Ferdinand Teuber, Damien Pennetreau, Margherita Bussi, Cal Le Gall","doi":"10.1177/00491241211067508","DOIUrl":"https://doi.org/10.1177/00491241211067508","url":null,"abstract":"Qualitative secondary analysis has generated heated debate regarding the epistemology of qualitative research. We argue that shifting to an abductive approach provides a fruitful avenue for qualitative secondary analysts who are oriented towards theory-building. However, the concrete implementation of abduction remains underdeveloped—especially for coding. We address this key gap by outlining a set of tactics for abductive analysis that can be applied for qualitative analysis. Our approach applies Timmermans and Tavory's ( Timmermans and Tavory 2012 ; Tavory and Timmermans 2014 ) three stages of abduction in three steps for qualitative (secondary) analysis: Generating an Abductive Codebook, Abductive Data Reduction through Code Equations, and In-Depth Abductive Qualitative Analysis. A key contribution of our article is the development of “code equations”—defined as the combination of codes to operationalize phenomena that span individual codes. Code equations are an important resource for abduction and other qualitative approaches that leverage qualitative data to build theory.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":" ","pages":""},"PeriodicalIF":6.3,"publicationDate":"2022-02-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46997816","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"In Search of a Comparable Measure of Generalized Individual Religiosity in the World Values Survey","authors":"A. Remizova, M. Rudnev, E. Davidov","doi":"10.1177/00491241221077239","DOIUrl":"https://doi.org/10.1177/00491241221077239","url":null,"abstract":"Individual religiosity measures are used by researchers to describe and compare individuals and societies. However, the cross-cultural comparability of the measures has often been questioned but rarely empirically tested. In the current study, we examined the cross-national measurement invariance properties of generalized individual religiosity in the sixth wave of the World Values Survey. For the analysis, we used multiple group confirmatory factor analysis and alignment. Our results demonstrated that a theoretically driven measurement model was not invariant across all countries. We suggested four unidimensional measurement models and four overlapping groups of countries in which these measurement models demonstrated approximate invariance. The indicators that covered praying practices, importance of religion, and confidence in its institutions were more cross-nationally invariant than other indicators.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":" ","pages":""},"PeriodicalIF":6.3,"publicationDate":"2022-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46774458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Visual Design and Cognition in List-Style Open-Ended Questions in Web Probing","authors":"Katharina Meitinger, Tanja Kunz","doi":"10.1177/00491241221077241","DOIUrl":"https://doi.org/10.1177/00491241221077241","url":null,"abstract":"<p>Previous research reveals that the visual design of open-ended questions should match the response task so that respondents can infer the expected response format. Based on a web survey including specific probes in a list-style open-ended question format, we experimentally tested the effects of varying numbers of answer boxes on several indicators of response quality. Our results showed that using multiple small answer boxes instead of one large box had a positive impact on the number and variety of themes mentioned, as well as on the conciseness of responses to specific probes. We found no effect on the relevance of themes and the risk of item non-response. Based on our findings, we recommend using multiple small answer boxes instead of one large box to convey the expected response format and improve response quality in specific probes. This study makes a valuable contribution to the field of web probing, extends the concept of response quality in list-style open-ended questions, and provides a deeper understanding of how visual design features affect cognitive response processes in web surveys.</p>","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"44 38","pages":""},"PeriodicalIF":6.3,"publicationDate":"2022-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138506061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Comparing Single- and Multiple-Question Designs of Measuring Family Income in China Family Panel Studies","authors":"Qiong Wu, Li-na Gu","doi":"10.1177/00491241221077238","DOIUrl":"https://doi.org/10.1177/00491241221077238","url":null,"abstract":"Family income questions in general purpose surveys are usually collected with either a single-question summary design or a multiple-question disaggregation design. It is unclear how estimates from the two approaches agree with each other. The current paper takes advantage of a large-scale survey that has collected family income with both methods. With data from 14,222 urban and rural families in the 2018 wave of the nationally representative China Family Panel Studies, we compare the two estimates, and further evaluate factors that might contribute to the discrepancy. We find that the two estimates are loosely matched in only a third of all families, and most of the matched families have a simple income structure. Although the mean of the multiple-question estimate is larger than that of the single-question estimate, the pattern is not monotonic. At lower percentiles up till the median, the single-question estimate is larger, whereas the multiple-question estimate is larger at higher percentiles. Larger family sizes and more income sources contribute to higher likelihood of inconsistent estimates from the two designs. Families with wage income as the main income source have the highest likelihood of giving consistent estimates compared with all other families. In contrast, families with agricultural income or property income as the main source tend to have very high probability of larger single-question estimates. Omission of certain income components and rounding can explain over half of the inconsistencies with higher multiple-question estimates and a quarter of the inconsistencies with higher single-question estimates.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":" ","pages":""},"PeriodicalIF":6.3,"publicationDate":"2022-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47384693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}