{"title":"Untapped Potential: Designed Digital Trace Data in Online Survey Experiments","authors":"Erin Macke, Claire Daviss, Emma Williams-Baron","doi":"10.1177/00491241241268770","DOIUrl":"https://doi.org/10.1177/00491241241268770","url":null,"abstract":"Researchers have developed many uses for digital trace data, yet most online survey experiments continue to rely on attitudinal rather than behavioral measures. We argue that researchers can collect digital trace data during online survey experiments with relative ease, at modest costs, and to substantial benefit. Because digital trace data unobtrusively measure survey participants’ behaviors, they can be used to analyze digital outcomes of theoretical and empirical interest, while reducing the risk of social desirability bias. We demonstrate the feasibility and utility of collecting digital trace data during online survey experiments through two original studies. In both, participants evaluated interactive digital resumes designed to track participants’ clicks, mouse movements, and time spent on the resumes. This novel approach allowed us to better understand participants’ search for information and cognitive processing in hiring decisions. There is immense, untapped potential value in collecting digital trace data during online survey experiments and using it to address important sociological research questions.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"3 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2024-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141910238","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Handle with Care: A Sociologist’s Guide to Causal Inference with Instrumental Variables","authors":"Chris Felton, Brandon M. Stewart","doi":"10.1177/00491241241235900","DOIUrl":"https://doi.org/10.1177/00491241241235900","url":null,"abstract":"Instrumental variables (IV) analysis is a powerful, but fragile, tool for drawing causal inferences from observational data. Sociologists increasingly turn to this strategy in settings where unmeasured confounding between the treatment and outcome is likely. This paper reviews the assumptions required for IV and the consequences of violating them, focusing on sociological applications. We highlight three methodological problems IV faces: (i) identification bias, an asymptotic bias from assumption violations; (ii) estimation bias, a finite-sample bias that persists even when assumptions hold; and (iii) type-M error, the exaggeration of effect size given statistical significance. In each case, we emphasize how weak instruments exacerbate these problems and make results sensitive to minor violations of assumptions. We survey IV papers from top sociology journals, finding that assumptions often go unstated and robust uncertainty measures are rarely used. We provide a practical checklist to show how IV, despite its fragility, can still be useful when handled with care.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"30 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2024-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141910241","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Age, Period, and Cohort Analysis With Bounding and Interactions","authors":"Jiwon Lee","doi":"10.1177/00491241241266279","DOIUrl":"https://doi.org/10.1177/00491241241266279","url":null,"abstract":"This article uses the example of voter turnout in US presidential elections to compare two new methods for age, period, and cohort (APC) analysis: the APC interaction model and the APC bounding analysis. While discussing the formal, conceptual, and interpretive differences between the two methods, the analysis demonstrates how both methods can be used to generate distinct but complementary findings. Because the two methods take alternative positions on the appropriate cohort-effect estimands, the comparison underscores the importance of well-grounded conceptual foundations in APC analysis.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"41 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2024-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141880349","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Causal Decomposition Analysis With Time-Varying Mediators: Designing Individualized Interventions to Reduce Social Disparities","authors":"Soojin Park, Namhwa Lee, Rafael Quintana","doi":"10.1177/00491241241264562","DOIUrl":"https://doi.org/10.1177/00491241241264562","url":null,"abstract":"Causal decomposition analysis aims to identify risk factors (referred to as “mediators”) that contribute to social disparities in an outcome. Despite promising developments in causal decomposition analysis, current methods are limited to addressing a time-fixed mediator and outcome only, which has restricted our understanding of the causal mechanisms underlying social disparities. In particular, existing approaches largely overlook individual characteristics when designing (hypothetical) interventions to reduce disparities. To address this issue, we extend current longitudinal mediation approaches to the context of disparities research. Specifically, we develop a novel decomposition analysis method that addresses individual characteristics by (a) using optimal dynamic treatment regimes (DTRs) and (b) conditioning on a selective set of individual characteristics. Incorporating optimal DTRs into the design of interventions can be used to strike a balance between equity (reducing disparities) and excellence (improving individuals’ outcomes). We illustrate the proposed method using the High School Longitudinal Study data.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"24 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2024-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141768471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Corrigendum to “Individual Components of Three Inequality Measures for Analyzing Shapes of Inequality”","authors":"","doi":"10.1177/00491241241263701","DOIUrl":"https://doi.org/10.1177/00491241241263701","url":null,"abstract":"","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"84 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141448566","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Aprile D Benner, Shanting Chen, Celeste C Fernandez, Mark D Hayward
{"title":"The Potential for Using a Shortened Version of the Everyday Discrimination Scale in Population Research with Young Adults: A Construct Validation Investigation.","authors":"Aprile D Benner, Shanting Chen, Celeste C Fernandez, Mark D Hayward","doi":"10.1177/00491241211067512","DOIUrl":"10.1177/00491241211067512","url":null,"abstract":"<p><p>Discrimination is associated with numerous psychological health outcomes over the life course. The nine-item Everyday Discrimination Scale (EDS) is one of the most widely used measures of discrimination; however, this nine-item measure may not be feasible in large-scale population health surveys where a shortened discrimination measure would be advantageous. The current study examined the construct validity of a combined two-item discrimination measure adapted from the EDS by Add Health (<i>N</i> = 14,839) as compared to the full nine-item EDS and a two-item EDS scale (parallel to the adapted combined measure) used in the National Survey of American Life (NSAL; <i>N</i> = 1,111) and National Latino and Asian American Study (NLAAS) studies (<i>N</i> = 1,055). Results identified convergence among the EDS scales, with high item-total correlations, convergent validity, and criterion validity for psychological outcomes, thus providing evidence for the construct validity of the two-item combined scale. Taken together, the findings provide support for using this reduced scale in studies where the full EDS scale is not available.</p>","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"1 1","pages":"804-838"},"PeriodicalIF":6.3,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11136476/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41461461","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The gap-closing estimand: A causal approach to study interventions that close disparities across social categories.","authors":"Ian Lundberg","doi":"10.1177/00491241211055769","DOIUrl":"10.1177/00491241211055769","url":null,"abstract":"<p><p>Disparities across race, gender, and class are important targets of descriptive research. But rather than only describe disparities, research would ideally inform interventions to close those gaps. The gap-closing estimand quantifies how much a gap (e.g. incomes by race) would close if we intervened to equalize a treatment (e.g. access to college). Drawing on causal decomposition analyses, this type of research question yields several benefits. First, gap-closing estimands place categories like race in a causal framework without making them play the role of the treatment (which is philosophically fraught for non-manipulable variables). Second, gap-closing estimands empower researchers to study disparities using new statistical and machine learning estimators designed for causal effects. Third, gap-closing estimands can directly inform policy: if we sampled from the population and actually changed treatment assignments, how much could we close gaps in outcomes? I provide open-source software (the R package gapclosing) to support these methods.</p>","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"53 2","pages":"507-570"},"PeriodicalIF":6.5,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11823715/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143415331","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"How Valid Are Trust Survey Measures? New Insights From Open-Ended Probing Data and Supervised Machine Learning","authors":"Camille Landesvatter, Paul C. Bauer","doi":"10.1177/00491241241234871","DOIUrl":"https://doi.org/10.1177/00491241241234871","url":null,"abstract":"Trust is a foundational concept of contemporary sociological theory. Still, empirical research on trust relies on a relatively small set of measures. These are increasingly debated, potentially undermining large swathes of empirical evidence. Drawing on a combination of open-ended probing data, supervised machine learning, and a U.S. representative quota sample, our study compares the validity of standard measures of generalized social trust with more recent, situation-specific measures of trust. We find that survey measures that refer to “strangers” in their question wording best reflect the concept of generalized trust, also known as trust in unknown others. While situation-specific measures should have the desirable property of further reducing variation in associations, that is, producing more similar frames of reference across respondents, they also seem to increase associations with known others, which is undesirable. In addition, we explore to what extent trust survey questions may evoke negative associations. We find that there is indeed variation across measures, which calls for more research.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"78 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2024-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140192831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Data Imbalances in Coincidence Analysis: A Simulation Study","authors":"Martyna Daria Swiatczak, Michael Baumgartner","doi":"10.1177/00491241241227039","DOIUrl":"https://doi.org/10.1177/00491241241227039","url":null,"abstract":"In this paper, we investigate the conditions under which data imbalances, a common data characteristic that occurs when factor values are unevenly distributed, are problematic for the performance of Coincidence Analysis (CNA). We further examine how such imbalances relate to fragmentation and noise in data. We show that even extreme data imbalances, when not combined with fragmentation or noise, do not negatively affect CNA’s performance. However, an extended series of simulation experiments on fuzzy-set data reveals that, when mixed with fragmentation or noise, data imbalances may substantially impair CNA’s performance. Furthermore, we find that the performance impairment is higher when endogenous factors are imbalanced than when exogenous factors are concerned. Our results allow us to quantify these impacts and demarcate degrees at which data imbalances should be considered as problematic. Thus, applied researchers can use our demarcation guidelines to enhance the validity of their studies.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"27 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2024-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140165080","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dustin S. Stoltz, Marshall A. Taylor, Jennifer S. K. Dudley
{"title":"A Tool Kit for Relation Induction in Text Analysis","authors":"Dustin S. Stoltz, Marshall A. Taylor, Jennifer S. K. Dudley","doi":"10.1177/00491241241233242","DOIUrl":"https://doi.org/10.1177/00491241241233242","url":null,"abstract":"Distances derived from word embeddings can measure a range of gradational relations—similarity, hierarchy, entailment, and stereotype—and can be used at the document- and author-level in ways that overcome some of the limitations of weighted dictionary methods. We provide a comprehensive introduction to using word embeddings for relation induction, and demonstrate how such techniques can complement dictionary methods as unsupervised, deductive methods.","PeriodicalId":21849,"journal":{"name":"Sociological Methods & Research","volume":"46 1","pages":""},"PeriodicalIF":6.3,"publicationDate":"2024-02-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140015572","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}