The semantics of sketch: Flexibility in visual query systems for time series data

Sketching allows analysts to specify complex and free-form patterns of interest. Visual query systems can make use of sketches to locate these patterns of interest in large datasets. However, sketching is ambiguous: the same drawing could represent a multitude of potential queries. In this work, we investigate these ambiguities as they apply to visual query systems for time series data. We define a class of “invariants” — the properties of a time series that the analyst wishes to ignore when performing a sketch-based query. We present the results of a crowd-sourced study, showing that these invariants are key components of how people rate the strength of match between sketch and target. We adapt a number of algorithms for time series matching to support invariants in sketches. Lastly, we present a web-deployed prototype sketch-based visual query system that relies on these invariants. We apply the prototype to data from finance, the digital humanities, and political science.