Online Handwritten Mathematical Expressions Recognition by Merging Multiple 1D Interpretations

Abstract

In this work, we propose to recognize handwritten mathematical expressions by merging multiple 1D sequences of labels produced by a sequence labeler. The proposed solution aims at rebuilding a 2D expression from several 1D labeled paths. An online math expression is a sequence of strokes which is later used to build a graph considering both temporal and spatial orders among these strokes. In this graph, node corresponds to stroke and edge denotes the relationship between a pair of strokes. Next, we select 1D paths from the built graph with the expectation that these paths could catch all the strokes and the relationships between pairs of strokes. As an advanced and strong sequence classifier, BLSTM networks are adopted to label the selected 1D paths. We set different weights to these 1D labeled paths and then merge them to rebuild a label graph. After that, an additional post-process will be performed to complete the edges automatically. We test the proposed solution and compare the results to the state of art in online math expression recognition domain.