Enhancing Code Transformation in Large Language Models Through Retrieval-Augmented Fine-Tuning

IF 4.3 2区 计算机科学 Q1 ENGINEERING, ELECTRICAL & ELECTRONIC
Jing-Ming Guo;Po-Yang Liu;Yi-Chong Zeng;Ting-Ju Chen
{"title":"Enhancing Code Transformation in Large Language Models Through Retrieval-Augmented Fine-Tuning","authors":"Jing-Ming Guo;Po-Yang Liu;Yi-Chong Zeng;Ting-Ju Chen","doi":"10.1109/TCE.2025.3565294","DOIUrl":null,"url":null,"abstract":"Large language models (LLMs) have made substantial advancements in knowledge reasoning and are increasingly utilized in specialized domains such as code completion, legal analysis, and medical transcription, where accuracy is paramount. In such applications, document-specific precision is more critical than general reasoning capabilities. This paper proposes a novel approach based on Retrieval-Augmented Fine-Tuning (RAFT) to enhance model-generated outputs, particularly in code transformation tasks. RAFT integrates domain-specific knowledge, optimizing in-domain retrieval-augmented generation by training the model to discern the relationship between prompts, retrieved documents, and target outputs. This enables the model to extract relevant information while mitigating the impact of noise. Experimental results demonstrate that the proposed method improves accuracy of 2.4% and CodeBLEU of 1.3% for VB-to-C# code conversion, highlighting its effectiveness in domain-specific applications.","PeriodicalId":13208,"journal":{"name":"IEEE Transactions on Consumer Electronics","volume":"71 1","pages":"2342-2346"},"PeriodicalIF":4.3000,"publicationDate":"2025-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Consumer Electronics","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10979988/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

Abstract

Large language models (LLMs) have made substantial advancements in knowledge reasoning and are increasingly utilized in specialized domains such as code completion, legal analysis, and medical transcription, where accuracy is paramount. In such applications, document-specific precision is more critical than general reasoning capabilities. This paper proposes a novel approach based on Retrieval-Augmented Fine-Tuning (RAFT) to enhance model-generated outputs, particularly in code transformation tasks. RAFT integrates domain-specific knowledge, optimizing in-domain retrieval-augmented generation by training the model to discern the relationship between prompts, retrieved documents, and target outputs. This enables the model to extract relevant information while mitigating the impact of noise. Experimental results demonstrate that the proposed method improves accuracy of 2.4% and CodeBLEU of 1.3% for VB-to-C# code conversion, highlighting its effectiveness in domain-specific applications.
通过检索增强微调增强大型语言模型中的代码转换
大型语言模型(llm)在知识推理方面取得了重大进展,并越来越多地用于代码完成、法律分析和医学转录等专业领域,这些领域的准确性至关重要。在这样的应用程序中,特定于文档的精度比一般推理能力更为重要。本文提出了一种基于检索增强微调(RAFT)的新方法来增强模型生成的输出,特别是在代码转换任务中。RAFT集成了领域特定的知识,通过训练模型来识别提示、检索文档和目标输出之间的关系来优化领域内检索增强生成。这使模型能够提取相关信息,同时减轻噪声的影响。实验结果表明,该方法在vb - c#代码转换中准确率提高了2.4%,codeleu提高了1.3%,突出了其在特定领域应用中的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
7.70
自引率
9.30%
发文量
59
审稿时长
3.3 months
期刊介绍: The main focus for the IEEE Transactions on Consumer Electronics is the engineering and research aspects of the theory, design, construction, manufacture or end use of mass market electronics, systems, software and services for consumers.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信