Long context window-based zero-shot legal interpretation of building codes and regulations

Abstract

South Korean authorities handle over 2000 inquiries daily about building code violations. Interpreting these complex, frequently updated codes is challenging, even for legal experts. Prior studies using large language models (LLMs) with retrieval-augmented generation (RAG) have struggled with context loss due to data segmentation. This paper proposes three automated building code interpreter (ABCI) models—Original, Inferred, and Filtered—that leverage long-context window (LCW) LLMs as the base model. On 171 challenging legal interpretative question-answering (LIQA) cases, ABCI-Filtered achieved 63.2 % accuracy, outperforming the RAG baseline approach (56.1 %), state-of-the-art LLMs like Claude 3.7 (60.2 %), as well as ABCI-Inferred (60.8 %) and ABCI-Original (56.7 %). Notably, unlike prior methods that require fine-tuning, ABCI-Filtered outperformed previous methods using only zero-shot reasoning. In an additional experiment using a relatively straightforward building code QA dataset, ABCI-Filtered and ABCI-Inferred outperformed the other methods (79.6 % and 80.0 %, respectively), confirming the difficulty of the initial task using the LIQA dataset.