LLM-based Approach to Automatically Establish Traceability between Requirements and MBSE

Abstract

Tracing requirements specification to design and implementation is an essential part of safety standards, as it allows to ensure that safety goals are met throughout the development process. Manual tracing numerous artifacts produced throughout the development process is error-prone and takes much time. To address these problems, we proposed a tool (Bonner, M.; Zeller, M.; Schulz, G.; Beyer, D.; Olteanu, M., 2023), which allows to establish links between requirements and Model-Based Systems Engineering (MBSE) in a semi-automatic way. The underlying algorithms of our tool are embedding similarity computation and classification approaches based on Large Language Models (LLMs). To assess the performance of underlying algorithm we propose an evaluation, where we compare the recall, the precision, and the F₂ score of different approaches applied to our datasets. The goal of our evaluation is to understand how well LLMs perform in automatically generating trace links on different datasets. Our evaluation shows that it is worth to invest time in preprocessing the data and fine-tuning the LLMs to achieve the better recommendations for engineers, which improves the traceability process.