Just-in-time Parsing with Scannerless Earley Virtual Machines

Abstract

Scannerless Earley Virtual Machines (SEVM) is a new generalized context-free parsing method, in which grammars are internally encoded using a special instruction intermediate language. In this paper we show how just-in-time compilation can be used to translate intermediate form grammars into native machine code to achieve improved parsing performance. We also present an efficient method for lexical disambiguation, which additionally enables to significantly reduce the amount of code that needs to be just-in-time compiled. Finally, we compare our implementation of SEVM with other parser implementations and show that our parser provides acceptable performance for analysing real-world computer languages.