On-line handwritten formula recognition using hidden Markov models and context dependent graph grammars

Abstract

This paper presents an approach for the recognition of on-line handwritten mathematical expressions. The hidden Markov model (HMM) based system makes use of simultaneous segmentation and recognition capabilities, avoiding a crucial segmentation during pre-processing. With the segmentation and recognition results, obtained from the HMM recognizer it is possible to analyze and interpret the spatial two-dimensional arrangement of the symbols. We use a graph grammar approach for the structure recognition, also used in off-line recognition process, resulting in a general tree-structure of the underlying input-expression. The resulting constructed tree can be translated to any desired syntax (for example: Lisp, KT/sub E/X, and OpenMath).