CRICTO: Supporting Sensemaking through Crowdsourced Information Schematization

Abstract

We present CRICTO, a new crowdsourcing visual analytics environment for making sense of and analyzing text data, whereby multiple crowdworkers are able to parallelize the simple information schematization tasks of relating and connecting entities across documents. The diverse links from these schematization tasks are then automatically combined and the system visualizes them based on the semantic types of the linkages. CRICTO also includes several tools that allow analysts to interactively explore and refine crowdworkers’ results to better support their own sensemaking processes. We evaluated CRICTO’s techniques and analysis workflow with deployments of CRICTO using Amazon Mechanical Turk and a user study that assess the effect of crowdsourced schematization in sensemaking tasks. The results of our evaluation show that CRICTO’s crowdsourcing approaches and workflow help analysts explore diverse aspects of datasets, and uncover more accurate hidden stories embedded in the text datasets.