KnowBug: Enhancing Large language models with bug report knowledge for deep learning framework bug prediction

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems Pub Date : 2024-10-10 DOI:10.1016/j.knosys.2024.112588

Chenglong Li , Zheng Zheng , Xiaoting Du , Xiangyue Ma , Zhengqi Wang , Xinheng Li

{"title":"KnowBug: Enhancing Large language models with bug report knowledge for deep learning framework bug prediction","authors":"Chenglong Li , Zheng Zheng , Xiaoting Du , Xiangyue Ma , Zhengqi Wang , Xinheng Li","doi":"10.1016/j.knosys.2024.112588","DOIUrl":null,"url":null,"abstract":"<div><div>Understanding and predicting the bug type is crucial for developers striving to enhance testing efficiency and reduce software release problems. Bug reports, although semi-structured, contain valuable semantic information, making their comprehension critical for accurate bug prediction. Recent advances in large language models (LLMs), especially generative LLMs, have demonstrated their power in natural language processing. Many studies have utilized these models to understand various forms of textual data. However, the capability of LLMs to fully understand bug reports remains uncertain. To tackle this challenge, we propose <em>KnowBug</em>, a framework designed to augment LLMs with knowledge from bug reports to improve their ability to predict bug types. In this framework, we utilize bug reports from open-source deep learning frameworks, design specialized prompts, and fine-tune LLMs to assess <em>KnowBug</em>’s proficiency in understanding bug reports and predicting different bug types.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2000,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Knowledge-Based Systems","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S095070512401222X","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

Understanding and predicting the bug type is crucial for developers striving to enhance testing efficiency and reduce software release problems. Bug reports, although semi-structured, contain valuable semantic information, making their comprehension critical for accurate bug prediction. Recent advances in large language models (LLMs), especially generative LLMs, have demonstrated their power in natural language processing. Many studies have utilized these models to understand various forms of textual data. However, the capability of LLMs to fully understand bug reports remains uncertain. To tackle this challenge, we propose KnowBug, a framework designed to augment LLMs with knowledge from bug reports to improve their ability to predict bug types. In this framework, we utilize bug reports from open-source deep learning frameworks, design specialized prompts, and fine-tune LLMs to assess KnowBug’s proficiency in understanding bug reports and predicting different bug types.

查看原文本刊更多论文

KnowBug：利用错误报告知识增强大型语言模型，用于深度学习框架的错误预测

对于努力提高测试效率和减少软件发布问题的开发人员来说，理解和预测错误类型至关重要。错误报告虽然是半结构化的，但包含有价值的语义信息，因此对它们的理解对于准确预测错误至关重要。大型语言模型（LLM），尤其是生成式 LLM 的最新进展已经证明了它们在自然语言处理方面的强大功能。许多研究都利用这些模型来理解各种形式的文本数据。然而，LLMs 完全理解错误报告的能力仍不确定。为了应对这一挑战，我们提出了 KnowBug，这是一个旨在利用错误报告中的知识增强 LLM 的框架，以提高它们预测错误类型的能力。在这个框架中，我们利用开源深度学习框架中的错误报告，设计专门的提示，并对 LLM 进行微调，以评估 KnowBug 在理解错误报告和预测不同错误类型方面的能力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Knowledge-Based Systems 工程技术-计算机：人工智能

CiteScore

14.80

自引率

12.50%

发文量

1245

审稿时长

7.8 months

期刊介绍： Knowledge-Based Systems, an international and interdisciplinary journal in artificial intelligence, publishes original, innovative, and creative research results in the field. It focuses on knowledge-based and other artificial intelligence techniques-based systems. The journal aims to support human prediction and decision-making through data science and computation techniques, provide a balanced coverage of theory and practical study, and encourage the development and implementation of knowledge-based intelligence models, methods, systems, and software tools. Applications in business, government, education, engineering, and healthcare are emphasized.