UNA: Improving Automated PL-NL System by A Unified Neural Architecture

Abstract

With the extensive application of artificial intelligence (AI) technologies, automated programming language-natural language (PL-NL) systems have gained significant attention, driving a series of related tasks served for developers and users, such as code search and summarization. Currently, mainstream PL-NL systems regard PL-NL as bimodal data and utilize two individual neural architectures (e.g., recurrent neural network) to learn the representation of PL-NL and build their semantic relations, improving the effects of these tasks. However, there exist two issues that limit the ability of these service systems in representation learning: first, large vocabularies cause data sparsity problems and limit the learning ability of neural architectures; second, there is not always a one-to-one correspondence between source code and natural language. To address these two issues, in this article, we introduce the unified neural architecture (UNA) by building a unified vocabulary (Uni-Vocab) at the subword level, to provide high-quality PL-NL services. In the Uni-Vocab, we build a unified modal encoding for PL-NL, which allows us to effectively control the vocabulary size and solve the data sparsity problem. Afterward, our built UNA can learn the unified contextual representation of PL-NL, which helps build their unified semantic relations. To validate the effectiveness of the proposed UNA, we perform experiments on code search and code summarization, which are two PL-NL tasks for developers and users. Experimental results demonstrate UNA can obtain noteworthy performance improvement. In detail, the baseline approaches in these two tasks get improvements by up to 36.09% and 18.02% in terms of mean reciprocal rank and bilingual evaluation understudy, respectively.