Line-of-Sight Stroke Graphs and Parzen Shape Context Features for Handwritten Math Formula Representation and Symbol Segmentation

Abstract

This paper presents a new representation for handwritten math formulae: a Line-of-Sight (LOS) graph over handwritten strokes, computed using stroke convex hulls. Experimental results using the CROHME 2012 and 2014 datasets show that LOS graphs capture the visual structure of handwritten formulae better than commonly used graphs such as Time-series, Minimum Spanning Trees, and k-Nearest Neighbor graphs. We then introduce a shape context-based feature (Parzen window Shape Contexts (PSC)) which is combined with simple geometric features and the distance in time between strokes to obtain state-of-the-art symbol segmentation results (92.43% F-measure for CROHME 2014). This result is obtained using a simple method, without use of OCR or an expression grammar. A binary random forest classifier identifies which LOS graph edges represent stroke pairs that should be merged into symbols, with connected components over merged strokes defining symbols. Line-of-Sight graphs and Parzen Shape Contexts represent visual structure well, and might be usefully applied to other notations.