Crowdster: enabling social navigation in web-based visualization using crowdsourced evaluation

Abstract

Evaluation is typically seen as a validation tool for visualization, but the proliferation of web-based visualization is enabling a radical new approach that uses crowdsourced evaluation for emergent collaboration where one user's efforts facilitate a crowd of future users. The idea is simple: instead of using clickstreams, keyboard input, and interaction logs to collect performance metrics for individual participants in a user study, the interaction data is aggregated from the running visualization, integrated back into the visual representation, and then the new interaction data is collected and evaluated with the old data. Known as social navigation, this enables users to build on the work of previous users, for example by seeing collective annotations, the most commonly selected data points, and the most popular locations on the visual space. However, while web-based visualizations by definition are distributed using a web server, most do not maintain the server-side database connections and aggregation mechanisms to achieve this. To bridge this gap between social navigation, its evaluation and visualization, we present Crowdster, a framework that supports capturing, aggregating, and visualizing user interaction data. We give three examples to showcase the Crowdster framework: a Google Maps app that shows the navigation trails of previous users, a scatterplot matrix that visualizes a density distribution of the most selected data points, and a node-link visualization that supports collective graph layout.