Lexical analyzer based on a self-organizing feature map

Abstract

In a lexical analyzer, the scanning of a whole dictionary using an editing distance measure has a very high computational cost. We present a lexical analyzer designed to focus on a very limited subset of the whole dictionary. The system is based on a self-organizing feature map which maps the dictionary on to a 2D space. The neighborhood relationships on this space are then used to define a short list of hypotheses. We introduce a multi-stage pyramidal network to speed up the access, and we present the performance of the system. These results are then interpreted.