A Replication Framework for Observational Research: From Specification Restriction to Specification Expansion

Abstract

Sociology has remained relatively insulated from debates about the “replication crisis.” Heeding calls to consider replication more deeply, we introduce a distinction between two types of research reforms that have emerged in the wake of the crisis: specification-restricting and specification-expanding reforms. Specification restriction is often promoted as a way to increase the repeatability of research findings by controlling false positives. We argue that this statistical justification is less compelling in observational contexts—where the assumptions licensing false-positive control are harder to sustain and rigid ex ante analytical commitments can instead enshrine fragile specifications—than in experimental contexts. We further argue that apparently low replication rates do not necessarily imply a high prevalence of false positive findings. Instead, we propose a replication framework centered on specification-expanding reforms, stronger incentives for confirmatory research, and meta-analysis. This approach equips sociologists to assess the repeatability of findings and build a more cumulative discipline.