Sean Kates, Tine Paulsen, Sidak Yntiso, Joshua A. Tucker
{"title":"Bridging the Grade Gap: Reducing Assessment Bias in a Multi-Grader Class","authors":"Sean Kates, Tine Paulsen, Sidak Yntiso, Joshua A. Tucker","doi":"10.1017/pan.2022.27","DOIUrl":"https://doi.org/10.1017/pan.2022.27","url":null,"abstract":"Abstract Many large survey courses rely on multiple professors or teaching assistants to judge student responses to open-ended questions. Even following best practices, students with similar levels of conceptual understanding can receive widely varying assessments from different graders. We detail how this can occur and argue that it is an example of differential item functioning (or interpersonal incomparability), where graders interpret the same possible grading range differently. Using both actual assessment data from a large survey course in Comparative Politics and simulation methods, we show that the bias can be corrected by a small number of “bridging” observations across graders. We conclude by offering best practices for fair assessment in large survey courses.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":"31 1","pages":"642 - 650"},"PeriodicalIF":5.4,"publicationDate":"2022-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47404335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Automated Coding of Political Campaign Advertisement Videos: An Empirical Validation Study","authors":"Alexander Tarr, June Hwang, Kosuke Imai","doi":"10.1017/pan.2022.26","DOIUrl":"https://doi.org/10.1017/pan.2022.26","url":null,"abstract":"Abstract Video advertisements, either through television or the Internet, play an essential role in modern political campaigns. For over two decades, researchers have studied television video ads by analyzing the hand-coded data from the Wisconsin Advertising Project and its successor, the Wesleyan Media Project (WMP). Unfortunately, manually coding more than a hundred of variables, such as issue mentions, opponent appearance, and negativity, for many videos is a laborious and expensive process. We propose to automatically code campaign advertisement videos. Applying state-of-the-art machine learning methods, we extract various audio and image features from each video file. We show that our machine coding is comparable to human coding for many variables of the WMP datasets. Since many candidates make their advertisement videos available on the Internet, automated coding can dramatically improve the efficiency and scope of campaign advertisement research. Open-source software package is available for implementing the proposed methodology.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":"31 1","pages":"554 - 574"},"PeriodicalIF":5.4,"publicationDate":"2022-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43838742","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Consequences of Model Misspecification for the Estimation of Nonlinear Interaction Effects","authors":"Janina Beiser-McGrath, Liam F. Beiser-McGrath","doi":"10.1017/pan.2022.25","DOIUrl":"https://doi.org/10.1017/pan.2022.25","url":null,"abstract":"Abstract Recent research has shown that interaction effects may often be nonlinear (Hainmueller, Mummolo, and Xu [2019, Political Analysis 27, 163–192]). As standard interaction effect specifications assume a linear interaction effect, that is, the moderator conditions the effect at a constant rate, this can lead to bias. However, allowing nonlinear interaction effects, without accounting for other nonlinearities and nonlinear interaction effects, can also lead to biased estimates. Specifically, researchers can infer nonlinear interaction effects, even though the true interaction effect is linear, when variables used for covariate adjustment that are correlated with the moderator have a nonlinear effect upon the outcome of interest. We illustrate this bias with simulations and show how diagnostic tools recommended in the literature are unable to uncover the issue. We show how using the adaptive Lasso to identify relevant nonlinearities among variables used for covariate adjustment can avoid this issue. Moreover, the use of regularized estimators, which allow for a fuller set of nonlinearities, both independent and interactive, is more generally shown to avoid this bias and more general forms of omitted interaction bias.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":"31 1","pages":"278 - 287"},"PeriodicalIF":5.4,"publicationDate":"2022-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45197888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Measuring Swing Voters with a Supervised Machine Learning Ensemble","authors":"Christopher Hare, Mikayla Kutsuris","doi":"10.1017/pan.2022.24","DOIUrl":"https://doi.org/10.1017/pan.2022.24","url":null,"abstract":"Abstract Theory has long suggested that swing voting is a response to cross-pressures arising from a mix of individual attributes and contextual factors. Unfortunately, existing regression-based approaches are ill-suited to explore the complex combinations of demographic, policy, and political factors that produce swing voters in American elections. This gap between theory and practice motivates our use of an ensemble of supervised machine learning methods to predict swing voters in the 2012, 2016, and 2020 U.S. presidential elections. The results from the learning ensemble substantiate the existence of swing voters in contemporary American elections. Specifically, we demonstrate that the learning ensemble produces well-calibrated and externally valid predictions of swing voter propensity in later elections and for related behaviors such as split-ticket voting. Although interpreting black-box models is more challenging, they can nonetheless provide meaningful substantive insights meriting further exploration. Here, we use flexible model-agnostic tools to perturb the ensemble and demonstrate that cross-pressures (particularly those involving ideological and policy-related considerations) are essential to accurately predict swing voters.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":"31 1","pages":"537 - 553"},"PeriodicalIF":5.4,"publicationDate":"2022-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46221099","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Change-Point Detection and Regularization in Time Series Cross-Sectional Data Analysis","authors":"J. Park, S. Yamauchi","doi":"10.1017/pan.2022.23","DOIUrl":"https://doi.org/10.1017/pan.2022.23","url":null,"abstract":"Abstract Researchers of time series cross-sectional data regularly face the change-point problem, which requires them to discern between significant parametric shifts that can be deemed structural changes and minor parametric shifts that must be considered noise. In this paper, we develop a general Bayesian method for change-point detection in high-dimensional data and present its application in the context of the fixed-effect model. Our proposed method, hidden Markov Bayesian bridge model, jointly estimates high-dimensional regime-specific parameters and hidden regime transitions in a unified way. We apply our method to Alvarez, Garrett, and Lange’s (1991, American Political Science Review 85, 539–556) study of the relationship between government partisanship and economic growth and Allee and Scalera’s (2012, International Organization 66, 243–276) study of membership effects in international organizations. In both applications, we found that the proposed method successfully identify substantively meaningful temporal heterogeneity in parameters of regression models.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":"31 1","pages":"257 - 277"},"PeriodicalIF":5.4,"publicationDate":"2022-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43366040","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Ordered Beta Regression: A Parsimonious, Well-Fitting Model for Continuous Data with Lower and Upper Bounds","authors":"R. Kubinec","doi":"10.1017/pan.2022.20","DOIUrl":"https://doi.org/10.1017/pan.2022.20","url":null,"abstract":"Abstract I propose a new model, ordered Beta regression, for continuous distributions with both lower and upper bounds, such as data arising from survey slider scales, visual analog scales, and dose–response relationships. This model employs the cut point technique popularized by ordered logit to fit a single linear model to both continuous (0,1) and degenerate [0,1] responses. The model can be estimated with or without observations at the bounds, and as such is a general solution for these types of data. Employing a Monte Carlo simulation, I show that the model is noticeably more efficient than ordinary least squares regression, zero-and-one-inflated Beta regression, rescaled Beta regression, and fractional logit while fully capturing nuances in the outcome. I apply the model to a replication of the Aidt and Jensen (2014, European Economic Review 72, 52–75) study of suffrage extensions in Europe. The model can be fit with the R package ordbetareg to facilitate hierarchical, dynamic, and multivariate modeling.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":"31 1","pages":"519 - 536"},"PeriodicalIF":5.4,"publicationDate":"2022-07-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42600884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kirk Bansak, Jens Hainmueller, D. Hopkins, Teppei Yamamoto
{"title":"Using Conjoint Experiments to Analyze Election Outcomes: The Essential Role of the Average Marginal Component Effect","authors":"Kirk Bansak, Jens Hainmueller, D. Hopkins, Teppei Yamamoto","doi":"10.1017/pan.2022.16","DOIUrl":"https://doi.org/10.1017/pan.2022.16","url":null,"abstract":"Abstract Political scientists have increasingly deployed conjoint survey experiments to understand multidimensional choices in various settings. In this paper, we show that the average marginal component effect (AMCE) constitutes an aggregation of individual-level preferences that is meaningful both theoretically and empirically. First, extending previous results to allow for arbitrary randomization distributions, we show how the AMCE represents a summary of voters’ multidimensional preferences that combines directionality and intensity according to a probabilistic generalization of the Borda rule. We demonstrate why incorporating both the directionality and intensity of multi-attribute preferences is essential for analyzing real-world elections, in which ceteris paribus comparisons almost never occur. Second, and in further empirical support of this point, we show how this aggregation translates directly into a primary quantity of interest to election scholars: the effect of a change in an attribute on a candidate’s or party’s expected vote share. These properties hold irrespective of the heterogeneity, strength, or interactivity of voters’ preferences and regardless of how votes are aggregated into seats. Finally, we propose, formalize, and evaluate the feasibility of using conjoint data to estimate alternative quantities of interest to electoral studies, including the effect of an attribute on the probability of winning.","PeriodicalId":48270,"journal":{"name":"Political Analysis","volume":"31 1","pages":"500 - 518"},"PeriodicalIF":5.4,"publicationDate":"2022-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44645179","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}