The Challenge of Quality in Social Computation

Abstract

Interactive web technologies now enable a host of so-called social computations, which can address challenges that are beyond the capabilities of machines alone. Notable examples of such social computation systems include Galaxy Zoo,1 BeeWatch,2 and Ushahidi,3 operating in fields as diverse as classification of newly discovered galaxies, monitoring of bee populations, and disaster management. A system for earthquake prediction using social media [Sakaki et al. 2010] illustrates how such computations can also emerge on social networking platforms. Social computations can be modeled as a complex collection of structured activities (i.e. workflows) that represent a blend of human and machine tasks, with associated objectives and reward mechanisms. In our previous work [Markovic et al. 2013; Markovic 2016] we argued that recording provenance of social computation workflows would enhance decision-making support for all associated stakeholders; these include initiators, participants, and beneficiaries of such computations. In the next section, we will briefly introduce the key characteristics of complex social computation systems before discussing why quality assessments in such a context are challenging.