Generating P4 data planes using LLMs

Abstract

Over the past few years, Large Language Models (LLMs) have become the source of impressive results in code generation. However, most research focuses on widely adopted general-purpose programming languages, with little attention given to niche domain-specific languages (DSLs). This raises the question: do DSLs, such as P4, a data plane programming language, have a place in the LLM world?

The potential impact of generating DSL code could be tremendous. Automatically generating data plane code promises flexible networks that can quickly adapt to specific conditions at the lowest level. P4 is structurally simpler than general-purpose languages, but also offers a much smaller corpus of existing programs, thus setting up interesting challenges for deep-learning based code generation.

In this paper, we show that crafting a highly specialized P4 dataset with domain knowledge is sufficient to bootstrap P4 code generation through fine-tuning existing LLMs, even when they have not encountered P4 code during pre-training. We further document the process of creating a relevant benchmark to assess the proficiency of fine-tuned models in generating P4 code. Our evaluation shows that our fine-tuned models outperform much larger models in both syntactic correctness and semantic alignment.