Disclosing sample bias fails to fully correct judgments of partisan extremity.

Abstract

How do we infer the beliefs of an entire group (e.g., Democrats) after being exposed to the beliefs of only a handful of group members? What if we know that the beliefs we encountered were selected in a biased manner? Across two experiments, we recruited 640 U.S. residents and assessed whether they could recognize and correct for such sample bias. Some participants viewed biased samples that exclusively featured the political opinions of extreme partisans, while others viewed representative samples free from selection biases. Results suggest that people do attempt to correct for known sample bias, but their efforts are often insufficient, leading them to make inaccurate inferences that align with sample bias. Specifically, participants tended to overestimate the ideological extremity of both Democrats and Republicans to a greater extent when exposed to explicitly biased samples, as opposed to representative ones. They also perceived members of the political party in question as holding more homogenous views, presumably because samples of extreme party members' views tend to have less variability than representative samples. Perhaps as a consequence, participants exposed to what they knew to be a biased sample, and who subsequently gave more biased estimates, did not express lower confidence in their estimates compared to participants who were shown representative samples. We discuss how a tendency to insufficiently adjust for transparently biased samples may contribute to partisan misperceptions that fuel political polarization.