More than a framework: Sketching out technical enablers for natural language-based source code generation

Abstract

Natural Language-based Source Code Generation (NLSCG) holds the promise to revolutionize the way how software is developed by means of facilitating a collection of intelligent technical enablers, based on sustained improvements on the natural language to source code pipelines and continuous adoption of new coding paradigms. In recent years, a large variety of NLSCG technical solutions have been proposed, and quite exciting experimental results have been reported. Meanwhile, current researches and initiative application projects in this area reflect a large diversity of NLSCG contexts and of major technical enablers. Such heterogeneity, fragmentation, and vagueness of the NLSCG technical landscape are currently frustrating the full realization of the NLSCG research and application vision. Players in this field could not find systematic guidelines on how to effectively address the ”known unknowns” and how to simply spot the ”unknown unknowns”, which eventually hinder the turning of NLSCG solutions into further research enhancements or production applications. Understanding the context, boundaries, capabilities, and integrations of NLSCG enablers is considered as one of the key drivers for the more practical application of NLSCG models. In this paper, we analyze in detail the natural language to source code pipelines and the evolvement of source code generation tasks, by considering both the problem context and technological aspects. A foresight reference framework for NLSCG is proposed to help handle the source code generation tasks with proper intelligent models. We review the present-day NLSCG technical landscape, as well as the core technical enablers along the source code generation pipelines. Relevant experiments are conducted to validate the role of representative models across different technical enablers on typical datasets, and we finally highlight the contribution of different enablers to code generation capabilities.