Political Analysis最新文献

Generalizing toward Nonrespondents: Effect Estimates in Survey Experiments Are Broadly Similar for Eager and Reluctant Participants 向非应答者推广：调查实验中的效果估计值对渴望参与和不愿参与的参与者大致相似

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-05-17 DOI: 10.1017/pan.2024.8

Philip Moniz, Rodrigo Ramirez-Perez, Erin Hartman, Stephen Jessee

{"title":"Generalizing toward Nonrespondents: Effect Estimates in Survey Experiments Are Broadly Similar for Eager and Reluctant Participants","authors":"Philip Moniz, Rodrigo Ramirez-Perez, Erin Hartman, Stephen Jessee","doi":"10.1017/pan.2024.8","DOIUrl":"https://doi.org/10.1017/pan.2024.8","url":null,"abstract":"\u0000 Survey experiments on probability samples are a popular method for investigating population-level causal questions due to their strong internal validity. However, lower survey response rates and an increased reliance on online convenience samples raise questions about the generalizability of survey experiments. We examine this concern using data from a collection of 50 survey experiments which represent a wide range of social science studies. Recruitment for these studies employed a unique double sampling strategy that first obtains a sample of “eager” respondents and then employs much more aggressive recruitment methods with the goal of adding “reluctant” respondents to the sample in a second sampling wave. This approach substantially increases the number of reluctant respondents who participate and also allows for straightforward categorization of eager and reluctant survey respondents within each sample. We find no evidence that treatment effects for eager and reluctant respondents differ substantially. Within demographic categories often used for weighting surveys, there is also little evidence of response heterogeneity between eager and reluctant respondents. Our results suggest that social science findings based on survey experiments, even in the modern era of very low response rates, provide reasonable estimates of population average treatment effects among a deeper pool of survey respondents in a wide range of settings.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":null,"pages":null},"PeriodicalIF":5.4,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141126441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Synthetic Replacements for Human Survey Data? The Perils of Large Language Models 人工调查数据的合成替代品？大型语言模型的危险

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-05-17 DOI: 10.1017/pan.2024.5

James Bisbee, Joshua D. Clinton, C. Dorff, Brenton Kenkel, Jennifer M. Larson

引用次数: 4

NonRandom Tweet Mortality and Data Access Restrictions: Compromising the Replication of Sensitive Twitter Studies 非随机推特死亡率和数据访问限制：影响敏感推特研究的复制

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-05-17 DOI: 10.1017/pan.2024.7

Andreas Küpfer

{"title":"NonRandom Tweet Mortality and Data Access Restrictions: Compromising the Replication of Sensitive Twitter Studies","authors":"Andreas Küpfer","doi":"10.1017/pan.2024.7","DOIUrl":"https://doi.org/10.1017/pan.2024.7","url":null,"abstract":"\u0000 Used by politicians, journalists, and citizens, Twitter has been the most important social media platform to investigate political phenomena such as hate speech, polarization, or terrorism for over a decade. A high proportion of Twitter studies of emotionally charged or controversial content limit their ability to replicate findings due to incomplete Twitter-related replication data and the inability to recrawl their datasets entirely. This paper shows that these Twitter studies and their findings are considerably affected by nonrandom tweet mortality and data access restrictions imposed by the platform. While sensitive datasets suffer a notably higher removal rate than nonsensitive datasets, attempting to replicate key findings of Kim’s (2023, Political Science Research and Methods 11, 673–695) influential study on the content of violent tweets leads to significantly different results. The results highlight that access to complete replication data is particularly important in light of dynamically changing social media research conditions. Thus, the study raises concerns and potential solutions about the broader implications of nonrandom tweet mortality for future social media research on Twitter and similar platforms.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":null,"pages":null},"PeriodicalIF":5.4,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140965760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Estimators for Topic-Sampling Designs 主题抽样设计的估计器

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-05-13 DOI: 10.1017/pan.2024.1

Scott Clifford, Carlisle Rainey

{"title":"Estimators for Topic-Sampling Designs","authors":"Scott Clifford, Carlisle Rainey","doi":"10.1017/pan.2024.1","DOIUrl":"https://doi.org/10.1017/pan.2024.1","url":null,"abstract":"\u0000 When researchers design an experiment, they usually hold potentially relevant features of the experiment constant. We call these details the “topic” of the experiment. For example, researchers studying the impact of party cues on attitudes must inform respondents of the parties’ positions on a particular policy. In doing so, researchers implement just one of many possible designs . Clifford, Leeper, and Rainey (2023. “Generalizing Survey Experiments Using Topic Sampling: An Application to Party Cues.” Forthcoming in Political Behavior. https://doi.org/10.1007/s11109-023-09870-1) argue that researchers should implement many of the possible designs in parallel—what they call “topic sampling”—to generalize to a larger population of topics. We describe two estimators for topic-sampling designs: First, we describe a nonparametric estimator of the typical effect that is unbiased under the assumptions of the design; and second, we describe a hierarchical model that researchers can use to describe the heterogeneity. We suggest describing the heterogeneity across topics in three ways: (1) the standard deviation in treatment effects across topics, (2) the treatment effects for particular topics, and (3) how the treatment effects for particular topics vary with topic-level predictors. We evaluate the performance of the hierarchical model using the Strengthening Democracy Challenge megastudy and show that the hierarchical model works well.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":null,"pages":null},"PeriodicalIF":5.4,"publicationDate":"2024-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140985053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Flexible Estimation of Policy Preferences for Witnesses in Committee Hearings 灵活估计委员会听证会证人的政策偏好

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-05-09 DOI: 10.1017/pan.2024.6

K. Esterling, Ju Yeon Park

引用次数: 0

Audit Experiments of Racial Discrimination and the Importance of Symmetry in Exposure to Cues 种族歧视的听觉实验和接触线索时对称性的重要性

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-05-06 DOI: 10.1017/pan.2024.3

Thomas Leavitt, Viviana Rivera-Burgos

{"title":"Audit Experiments of Racial Discrimination and the Importance of Symmetry in Exposure to Cues","authors":"Thomas Leavitt, Viviana Rivera-Burgos","doi":"10.1017/pan.2024.3","DOIUrl":"https://doi.org/10.1017/pan.2024.3","url":null,"abstract":"\u0000 Researchers are often interested in whether discrimination on the basis of racial cues persists above and beyond discrimination on the basis of nonracial attributes that decision makers—e.g., employers and legislators—infer from such cues. We show that existing audit experiments may be unable to parse these mechanisms because of an asymmetry in when decision makers are exposed to cues of race and additional signals intended to rule out discrimination due to other attributes. For example, email audit experiments typically cue race via the name in the email address, at which point legislators can choose to open the email, but cue other attributes in the body of the email, which decision makers can be exposed to only after opening the email. We derive the bias resulting from this asymmetry and then propose two distinct solutions for email audit experiments. The first exposes decision makers to all cues before the decision to open. The second crafts the email to ensure no discrimination in opening and then exposes decision makers to all cues in the body of the email after opening. This second solution works without measures of opening, but can be improved when researchers do measure opening, even if with error.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":null,"pages":null},"PeriodicalIF":5.4,"publicationDate":"2024-05-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141010354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Votes Can Be Confidently Bought in Some Ranked Ballot Elections, and What to Do about It 在某些排序选票选举中，选票很容易被买走，该怎么办？

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-05-06 DOI: 10.1017/pan.2024.4

Jack R. Williams, Samuel Baltz, Charles Stewart

引用次数: 0

Normal Mode Copulas for Nonmonotonic Dependence 非单调依赖性的正态模式协方差

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-02-13 DOI: 10.1017/pan.2023.45

Kentato Fukumoto

引用次数: 1

Normal Mode Copulas for Nonmonotonic Dependence 非单调依赖性的正态模式协方差

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-02-13 DOI: 10.1017/pan.2023.45

Kentato Fukumoto

引用次数: 1

Estimating the Ideology of Political YouTube Videos 估计 YouTube 政治视频的意识形态

IF 5.4 2区社会学

Political Analysis Pub Date : 2024-02-13 DOI: 10.1017/pan.2023.42

Angela Lai, Megan A. Brown, James Bisbee, Joshua A. Tucker, Jonathan Nagler, Richard Bonneau

{"title":"Estimating the Ideology of Political YouTube Videos","authors":"Angela Lai, Megan A. Brown, James Bisbee, Joshua A. Tucker, Jonathan Nagler, Richard Bonneau","doi":"10.1017/pan.2023.42","DOIUrl":"https://doi.org/10.1017/pan.2023.42","url":null,"abstract":"\u0000 We present a method for estimating the ideology of political YouTube videos. The subfield of estimating ideology as a latent variable has often focused on traditional actors such as legislators, while more recent work has used social media data to estimate the ideology of ordinary users, political elites, and media sources. We build on this work to estimate the ideology of a political YouTube video. First, we start with a matrix of political Reddit posts linking to YouTube videos and apply correspondence analysis to place those videos in an ideological space. Second, we train a language model with those estimated ideologies as training labels, enabling us to estimate the ideologies of videos not posted on Reddit. These predicted ideologies are then validated against human labels. We demonstrate the utility of this method by applying it to the watch histories of survey respondents to evaluate the prevalence of echo chambers on YouTube in addition to the association between video ideology and viewer engagement. Our approach gives video-level scores based only on supplied text metadata, is scalable, and can be easily adjusted to account for changes in the ideological landscape.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":null,"pages":null},"PeriodicalIF":5.4,"publicationDate":"2024-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139841347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2