Systematic Social Observation at Scale: Using Crowdsourcing and Computer Vision to Measure Visible Neighborhood Conditions

Analysis of neighborhood environments is important for understanding inequality. Few studies, however, use direct measures of the visible characteristics of neighborhood conditions, despite their theorized importance in shaping individual and community well-being, because collecting data on the physical conditions of places across neighborhoods and cities and over time has required extensive time and labor. The authors introduce systematic social observation at scale (SSO@S), a pipeline for using visual data, crowdsourcing, and computer vision to identify visible characteristics of neighborhoods at a large scale. The authors implement SSO@S on millions of street-level images across three physically distinct cities—Boston, Detroit, and Los Angeles—from 2007 to 2020 to identify trash across space and over time. The authors evaluate the extent to which this approach can be used to assist with systematic coding of street-level imagery through cross-validation and out-of-sample validation, class-activation mapping, and comparisons with other sources of observed neighborhood characteristics. The SSO@S approach produces estimates with high reliability that correlate with some expected demographic characteristics but not others, depending on the city. The authors conclude with an assessment of this approach for measuring visible characteristics of neighborhoods and the implications for methods and research.