{"title":"Post-Instrument Bias in Linear Models","authors":"Adam Glynn, M. Rueda, Julian Schuessler","doi":"10.1177/00491241231156965","DOIUrl":"https://doi.org/10.1177/00491241231156965","url":null,"abstract":"Post-instrument covariates are often included as controls in instrumental variable (IV) analyses to address a violation of the exclusion restriction. However, we show that such analyses are subject to biases unless strong assumptions hold. Using linear constant-effects models, we present asymptotic bias formulas for three estimators (with and without measurement error): IV with post-instrument covariates, IV without post-instrument covariates, and ordinary least squares. In large samples and when the model provides a reasonable approximation, these formulas sometimes allow the analyst to bracket the parameter of interest with two estimators and allow the analyst to choose the estimator with the least asymptotic bias. We illustrate these points with a discussion of the settler mortality IV used by Acemoglu, Johnson, and Robinson.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"85 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116921471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anders Holm, Anders Hjorth-Trolle, Robert Andersen
{"title":"Lagged Dependent Variable Predictors, Classical Measurement Error, and Path Dependency: The Conditions Under Which Various Estimators are Appropriate","authors":"Anders Holm, Anders Hjorth-Trolle, Robert Andersen","doi":"10.1177/00491241231176845","DOIUrl":"https://doi.org/10.1177/00491241231176845","url":null,"abstract":"Lagged dependent variables (LDVs) are often used as predictors in ordinary least squares (OLS) models in the social sciences. Although several estimators are commonly employed, little is known about their relative merits in the presence of classical measurement error and different longitudinal processes. We assess the performance of four commonly used estimators: (1) the standard OLS estimator, (2) an average of past measures (AVG), (3) an instrumental variable (IV) measured at one period previously (IV), and (4) an IV derived from information from more than one time before (IV2). We also propose a new estimator for fixed effects models—the first difference instrumental variable (FDIV) estimator. After exploring the consistency of these estimators, we demonstrate their performance using an empirical application predicting primary school test scores. Our results demonstrate that for a Markov process with classic measurement error (CME), IV and IV2 estimators are generally consistent; LDV and AVG estimators are not. For a semi-Markov process, only the IV2 estimator is consistent. On the other hand, if fixed effects are included in the model, only the FDIV estimator is consistent. We end with advice on how to select the appropriate estimator.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115170007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Effects of Open-Ended Probes on Closed Survey Questions in Web Surveys","authors":"Patrícia Hadler","doi":"10.1177/00491241231176846","DOIUrl":"https://doi.org/10.1177/00491241231176846","url":null,"abstract":"Probes are follow-ups to survey questions used to gain insights on respondents’ understanding of and responses to these questions. They are usually administered as open-ended questions, primarily in the context of questionnaire pretesting. Due to the decreased cost of data collection for open-ended questions in web surveys, researchers have argued for embedding more open-ended probes in large-scale web surveys. However, there are concerns that this may cause reactivity and impact survey data. The study presents a randomized experiment in which identical survey questions were run with and without open-ended probes. Embedding open-ended probes resulted in higher levels of survey break off, as well as increased backtracking and answer changes to previous questions. In most cases, there was no impact of open-ended probes on the cognitive processing of and response to survey questions. Implications for embedding open-ended probes into web surveys are discussed.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125676050","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploring and Correcting the Bias in the Estimation of the Gini Measure of Inequality","authors":"J. Muñoz, P. Moya-Fernández, E. Álvarez-Verdejo","doi":"10.1177/00491241231176847","DOIUrl":"https://doi.org/10.1177/00491241231176847","url":null,"abstract":"The Gini index is probably the most commonly used indicator to measure inequality. For continuous distributions, the Gini index can be computed using several equivalent formulations. However, this is not the case with discrete distributions, where controversy remains regarding the expression to be used to estimate the Gini index. We attempt to bring a better understanding of the underlying problem by regrouping and classifying the most common estimators of the Gini index proposed in both infinite and finite populations, and focusing on the biases. We use Monte Carlo simulation studies to analyse the bias of the various estimators under a wide range of scenarios. Extremely large biases are observed in heavy-tailed distributions with high Gini indices, and bias corrections are recommended in this situation. We propose the use of some (new and traditional) bootstrap-based and jackknife-based strategies to mitigate this bias problem. Results are based on continuous distributions often used in the modelling of income distributions. We describe a simulation-based criterion for deciding when to use bias corrections. Various real data sets are used to illustrate the practical application of the suggested bias corrected procedures.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125461399","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fieldwork Disrupted: How Researchers Adapt to Losing Access to Field Sites","authors":"Eric W. Schoon","doi":"10.1177/00491241231156961","DOIUrl":"https://doi.org/10.1177/00491241231156961","url":null,"abstract":"This article explores how researchers adapt to disruptions that cost them access to their field sites, advancing a uniquely sociological perspective on the dynamics of flexibility and adaptation in qualitative methods. Through interviews with 31 ethnographers whose access was preempted or eliminated, I find that adaptation varied systematically based on when during the fieldwork process researchers' access was disrupted. The timing of the disruption shaped the relevance and implications of common conditions that affect fieldwork, such as funding availability, institutionalized time constraints, and sunk costs. Consequently, despite a lack of common conventions or training in how to adapt to losing access, adaptations took one of three general forms, which I refer to as turning home, pivoting, and following. I highlight specific challenges associated with each of these forms and offer insights for navigating them. Building from my findings, I make the case that the logistics of being flexible and adapting are part of a hidden curriculum in qualitative methods, and I discuss how interrogating the conditions that structure these aspects of fieldwork advances research and pedagogy in qualitative methodology.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130450444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Longitudinal QCA: Integrating Time Through Change-Based Intervals (CBIs) and a Flexible Lag Condition (FLC)","authors":"C. Niessen","doi":"10.1177/00491241231156967","DOIUrl":"https://doi.org/10.1177/00491241231156967","url":null,"abstract":"In the wake of the methodological developments that aim to render qualitative comparative analysis (QCA) “time sensitive”, I propose a new procedure for carrying out QCA longitudinally. More specifically, I show first why longitudinal case disaggregation should be carried out with change-based intervals (CBIs) rather than with fixed intervals. Second, I develop a flexible lag condition (FLC) that (i) resolves two types of temporal contradictions and outcome redundancies that can result from temporal case disaggregation and (ii) allows to measure the average duration it takes for a combination of conditions to translate to an outcome. Since temporal contradictions and outcome redundancies are most likely with an increasing number of time points and conditions, as well as with CBIs in general, the FLC procedure is most useful in these cases. The fact that the interest of longitudinal analyses increases with the number of disaggregated cases underlines the usefulness of the proposed methodological innovation. Despite its suitability for mid-n and large-n analyses, longitudinal QCA with an FLC preserves a strong case-oriented and qualitative perspective and remains thereby loyal to QCA's original foundations.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114416358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Inter-Rater Reliability Methods in Qualitative Case Study Research","authors":"R. Cole","doi":"10.1177/00491241231156971","DOIUrl":"https://doi.org/10.1177/00491241231156971","url":null,"abstract":"The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to report them, despite some disclosure of multiple researcher teams and coding reconciliation in the work. The article argues that the in-depth discussion and reconciliation initiated by IRR may enhance the findings and theory that emerges from qualitative case study data analysis, where the main data source is often interview transcripts or field notes. To achieve this, the article provides a missing link in the literature between data gathering and analysis by expanding an existing process model from five to six stages. The article also identifies seven factors that researchers can consider to determine the suitability of IRR to their work and it offers an IRR checklist, thereby providing a contribution to the broader literature on qualitative research methods.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"100 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-02-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122606243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Estimating Causal Effects of Multi-Valued Treatments Accounting for Network Interference: Immigration Policies and Crime Rates","authors":"C. Tortú, I. Crimaldi, F. Mealli, L. Forastiere","doi":"10.1177/00491241221147503","DOIUrl":"https://doi.org/10.1177/00491241221147503","url":null,"abstract":"Policy evaluation studies, which assess the effect of an intervention, face statistical challenges: in real-world settings treatments are not randomly assigned and the analysis might be complicated by the presence of interference among units. Researchers have started to develop methods that allow to manage spillovers in observational studies; recent works focus primarily on binary treatments. However, many studies deal with more complex interventions. For instance, in political science, evaluating the impact of policies implemented by administrative entities often implies a multi-valued approach, as a policy towards a specific issue operates at many levels and can be defined along multiple dimensions. In this work, we extend the statistical framework about causal inference under network interference in observational studies, allowing for a multi-valued individual treatment and an interference structure shaped by a weighted network. The estimation strategy relies on a joint multiple generalized propensity score and allows one to estimate direct effects, controlling for both individual and network covariates. We follow this methodology to analyze the impact of the national immigration policy on the crime rate, analyzing data of 22 OECD countries over a thirty-years time frame. We define a multi-valued characterization of political attitude towards migrants and we assume that the extent to which each country can be influenced by another country is modeled by an indicator, summarizing their cultural and geographical proximity. Results suggest that implementing a highly restrictive immigration policy leads to an increase of the crime rate and the estimated effect is larger if we account for interference.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116526090","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Modeling the Bias of Digital Data: An Approach to Combining Digital With Official Statistics to Estimate and Predict Migration Trends","authors":"Yuan Hsiao, L. Fiorio, J. Wakefield, E. Zagheni","doi":"10.1177/00491241221140144","DOIUrl":"https://doi.org/10.1177/00491241221140144","url":null,"abstract":"Obtaining reliable and timely estimates of migration flows is critical for advancing the migration theory and guiding policy decisions, but it remains a challenge. Digital data provide granular information on time and space, but do not draw from representative samples of the population, leading to biased estimates. We propose a method for combining digital data and official statistics by using the official statistics to model the spatial and temporal dependence structure of the biases of digital data. We use simulations to demonstrate the validity of the model, then empirically illustrate our approach by combining geo-located Twitter data with data from the American Community Survey (ACS) to estimate state-level out-migration probabilities in the United States. We show that our model, which combines unbiased and biased data, produces predictions that are more accurate than predictions based solely on unbiased data. Our approach demonstrates how digital data can be used to complement, rather than replace, official statistics.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127633786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Multiple imputation of partially observed covariates in discrete-time survival analysis","authors":"Anna Haensch, J. Bartlett, Bernd Weiss","doi":"10.1177/00491241221140147","DOIUrl":"https://doi.org/10.1177/00491241221140147","url":null,"abstract":"Discrete-time survival analysis (DTSA) models are a popular way of modeling events in the social sciences. However, the analysis of discrete-time survival data is challenged by missing data in one or more covariates. Negative consequences of missing covariate data include efficiency losses and possible bias. A popular approach to circumventing these consequences is multiple imputation (MI). In MI, it is crucial to include outcome information in the imputation models. As there is little guidance on how to incorporate the observed outcome information into the imputation model of missing covariates in DTSA, we explore different existing approaches using fully conditional specification (FCS) MI and substantive-model compatible (SMC)-FCS MI. We extend SMC-FCS for DTSA and provide an implementation in the smcfcs R package. We compare the approaches using Monte Carlo simulations and demonstrate a good performance of the new approach compared to existing approaches.","PeriodicalId":286027,"journal":{"name":"Sociological Methods & Research","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115500334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}