Development of Lightweight Parsers with Different Go Language Granularity

Abstract

This article considers an approach to creating a family of lightweight grammars for the Go language, with a special symbol Any denoting the skipped part of the program [1]. A formal definition of a more granular grammar is given, along with examples illustrating the increase in grammar rule granularity. The efficiency of the constructed lightweight parsers is analyzed in terms of memory usage and runtime on seven industrial repositories. It is shown that increasing the granularity of the grammar does not lead to a significant increase in parser resource consumption and varies only slightly depending on repository type and coding style in Go. Furthermore, the advantages of using lightweight grammars with Any over full grammars are summarized. An example of using a lightweight grammar to determine code complexity is presented. The results can also be applied to estimate the parser’s contribution to overall resource consumption, e.g., in tasks such as code binding and project markup.