Application of retrieval-augmented generation for interactive industrial knowledge management via a large language model

IF 4.1 2区计算机科学 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Computer Standards & Interfaces Pub Date : 2025-03-06 DOI:10.1016/j.csi.2025.103995

Lun-Chi Chen , Mayuresh Sunil Pardeshi , Yi-Xiang Liao , Kai-Chih Pai

{"title":"Application of retrieval-augmented generation for interactive industrial knowledge management via a large language model","authors":"Lun-Chi Chen , Mayuresh Sunil Pardeshi , Yi-Xiang Liao , Kai-Chih Pai","doi":"10.1016/j.csi.2025.103995","DOIUrl":null,"url":null,"abstract":"<div><div>Industrial data processing and retrieval are necessary for adoption in Industry 5.0. Large Language Model (LLMs) revolutionize natural language process (NLP) but face challenges in domain-specific applications due to specialized terminology and context. Artificial Intelligence (AI) assistants for industrial-related work enquiry and customer support services are necessary for increasing demand and quality of service (QoS). Our research aims to design a novel customized model with a retrieval-augmented generation (RAG)-based LLM as a sustainable solution for industrial integration with AI. The goal is to provide an interactive industrial knowledge management (IIKM) system that can be applied to technical services: assisting technicians in the search for precise technical repair details and company internal regulation searches: personnel can easily inquire about regulations, such as business trips and leave requirements. The IIKM model architecture consists of BM25 and embedding sequence processing in the chroma database, where the top k-chunks are selected by the BAAI ranker to respond effectively to the queries. A group of documents of 234 MB size and pdf, pptx, docx, csv and txt formats are used for the experimental analysis. The designed interactive knowledge management system has a mean reciprocal rank (MRR) of 88 %, a recall of 85 % and a mean average precision (mAP) of 75 % in technical service. The internal regulatory documents have a generation-based retrieval evaluation prediction of recall of 91.62 %, MRR of 97.97 % and mAP of 91.12 %. We conclude with insights gained and experiences shared from IIKM deployment with Sakura incorporation, highlighting the importance of the hybrid approach integrating RAG-based generative pretrained transformer (GPT) models for customized solutions.</div></div>","PeriodicalId":50635,"journal":{"name":"Computer Standards & Interfaces","volume":"94 ","pages":"Article 103995"},"PeriodicalIF":4.1000,"publicationDate":"2025-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Standards & Interfaces","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0920548925000248","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}

引用次数: 0

Abstract

Industrial data processing and retrieval are necessary for adoption in Industry 5.0. Large Language Model (LLMs) revolutionize natural language process (NLP) but face challenges in domain-specific applications due to specialized terminology and context. Artificial Intelligence (AI) assistants for industrial-related work enquiry and customer support services are necessary for increasing demand and quality of service (QoS). Our research aims to design a novel customized model with a retrieval-augmented generation (RAG)-based LLM as a sustainable solution for industrial integration with AI. The goal is to provide an interactive industrial knowledge management (IIKM) system that can be applied to technical services: assisting technicians in the search for precise technical repair details and company internal regulation searches: personnel can easily inquire about regulations, such as business trips and leave requirements. The IIKM model architecture consists of BM25 and embedding sequence processing in the chroma database, where the top k-chunks are selected by the BAAI ranker to respond effectively to the queries. A group of documents of 234 MB size and pdf, pptx, docx, csv and txt formats are used for the experimental analysis. The designed interactive knowledge management system has a mean reciprocal rank (MRR) of 88 %, a recall of 85 % and a mean average precision (mAP) of 75 % in technical service. The internal regulatory documents have a generation-based retrieval evaluation prediction of recall of 91.62 %, MRR of 97.97 % and mAP of 91.12 %. We conclude with insights gained and experiences shared from IIKM deployment with Sakura incorporation, highlighting the importance of the hybrid approach integrating RAG-based generative pretrained transformer (GPT) models for customized solutions.

查看原文本刊更多论文

基于大语言模型的检索增强生成在交互式工业知识管理中的应用

工业数据处理和检索是在工业5.0中采用的必要条件。大型语言模型（llm）彻底改变了自然语言过程（NLP），但由于专业术语和上下文的限制，在特定领域的应用中面临挑战。工业相关工作查询和客户支持服务的人工智能（AI）助手对于增加需求和服务质量（QoS）是必要的。我们的研究旨在设计一种基于检索增强生成（RAG）的LLM的新型定制模型，作为工业与人工智能集成的可持续解决方案。目标是提供一个可应用于技术服务的交互式工业知识管理（IIKM）系统：协助技术人员查找精确的技术维修细节和公司内部法规查询：人员可以轻松查询法规，例如出差和休假要求。IIKM模型架构包括BM25和在色度数据库中的嵌入序列处理，其中BAAI排名器选择前k块以有效响应查询。实验分析使用了一组234mb大小的文档，分别为pdf、pptx、docx、csv和txt格式。所设计的交互式知识管理系统在技术服务中的平均互反秩（MRR）为88%，召回率为85%，平均平均精度（mAP）为75%。内部规范性文件基于生成的检索评价预测召回率为91.62%，MRR为97.97%，mAP为91.12%。最后，我们总结了Sakura公司从IIKM部署中获得的见解和经验，强调了将基于rag的生成预训练变压器（GPT）模型集成到定制解决方案中的混合方法的重要性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Computer Standards & Interfaces 工程技术-计算机：软件工程

CiteScore

11.90

自引率

16.00%

发文量

审稿时长

6 months

期刊介绍： The quality of software, well-defined interfaces (hardware and software), the process of digitalisation, and accepted standards in these fields are essential for building and exploiting complex computing, communication, multimedia and measuring systems. Standards can simplify the design and construction of individual hardware and software components and help to ensure satisfactory interworking. Computer Standards & Interfaces is an international journal dealing specifically with these topics. The journal • Provides information about activities and progress on the definition of computer standards, software quality, interfaces and methods, at national, European and international levels • Publishes critical comments on standards and standards activities • Disseminates user''s experiences and case studies in the application and exploitation of established or emerging standards, interfaces and methods • Offers a forum for discussion on actual projects, standards, interfaces and methods by recognised experts • Stimulates relevant research by providing a specialised refereed medium.