MaViLa: Unlocking new potentials in smart manufacturing through vision language models

IF 12.2 1区 工程技术 Q1 ENGINEERING, INDUSTRIAL
Haolin Fan , Chenshu Liu , Neville Elieh Janvisloo , Shijie Bian , Jerry Ying Hsi Fuh , Wen Feng Lu , Bingbing Li
{"title":"MaViLa: Unlocking new potentials in smart manufacturing through vision language models","authors":"Haolin Fan ,&nbsp;Chenshu Liu ,&nbsp;Neville Elieh Janvisloo ,&nbsp;Shijie Bian ,&nbsp;Jerry Ying Hsi Fuh ,&nbsp;Wen Feng Lu ,&nbsp;Bingbing Li","doi":"10.1016/j.jmsy.2025.02.017","DOIUrl":null,"url":null,"abstract":"<div><div>In smart manufacturing, there remains a gap in the system-level understanding of manufacturing processes that hinders the effective integration of artificial intelligence (AI) for autonomous planning and execution in dynamic real-world scenarios. This paper presents MaViLa, an advanced vision language model (VLM) specifically designed for the smart manufacturing domain. MaViLa enhances visual understanding in the manufacturing domain through two key approaches: first, it uses a retrieval augmented generation (RAG) pipeline to incorporate domain knowledge during dataset creation, and second, it implements a robust two-stage training paradigm of pre-training followed by instruction fine-tuning. Comparative evaluations of domain-relevant benchmarks demonstrate MaViLa’s superior performance over general-purpose VLMs, particularly in manufacturing-specific tasks such as process optimization and quality control. Experimental results, including laboratory tests and in-situ monitoring applications, highlight the effectiveness of MaViLa in scene understanding and decision-making support. With its scalability and seamless integration of external tools, MaViLa paves the way for more efficient human–machine interactions and the development of autonomous, holistic manufacturing systems. These advancements establish MaViLa as a key technology that unlocks new potential for smart manufacturing.</div></div>","PeriodicalId":16227,"journal":{"name":"Journal of Manufacturing Systems","volume":"80 ","pages":"Pages 258-271"},"PeriodicalIF":12.2000,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Manufacturing Systems","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0278612525000470","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, INDUSTRIAL","Score":null,"Total":0}
引用次数: 0

Abstract

In smart manufacturing, there remains a gap in the system-level understanding of manufacturing processes that hinders the effective integration of artificial intelligence (AI) for autonomous planning and execution in dynamic real-world scenarios. This paper presents MaViLa, an advanced vision language model (VLM) specifically designed for the smart manufacturing domain. MaViLa enhances visual understanding in the manufacturing domain through two key approaches: first, it uses a retrieval augmented generation (RAG) pipeline to incorporate domain knowledge during dataset creation, and second, it implements a robust two-stage training paradigm of pre-training followed by instruction fine-tuning. Comparative evaluations of domain-relevant benchmarks demonstrate MaViLa’s superior performance over general-purpose VLMs, particularly in manufacturing-specific tasks such as process optimization and quality control. Experimental results, including laboratory tests and in-situ monitoring applications, highlight the effectiveness of MaViLa in scene understanding and decision-making support. With its scalability and seamless integration of external tools, MaViLa paves the way for more efficient human–machine interactions and the development of autonomous, holistic manufacturing systems. These advancements establish MaViLa as a key technology that unlocks new potential for smart manufacturing.

Abstract Image

MaViLa:通过视觉语言模型释放智能制造的新潜力
在智能制造中,对制造过程的系统级理解仍然存在差距,这阻碍了人工智能(AI)在动态现实场景中的有效集成,以实现自主规划和执行。MaViLa是一种专门为智能制造领域设计的高级视觉语言模型(VLM)。MaViLa通过两种关键方法增强了制造领域的视觉理解:首先,它使用检索增强生成(RAG)管道在数据集创建过程中整合领域知识;其次,它实现了一种鲁棒的两阶段训练范式,即预训练和指令微调。领域相关基准的比较评估表明,MaViLa比通用vlm性能优越,特别是在工艺优化和质量控制等制造特定任务中。实验结果,包括实验室测试和现场监测应用,突出了MaViLa在场景理解和决策支持方面的有效性。凭借其可扩展性和与外部工具的无缝集成,MaViLa为更高效的人机交互和自主整体制造系统的开发铺平了道路。这些进步使MaViLa成为解锁智能制造新潜力的关键技术。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Manufacturing Systems
Journal of Manufacturing Systems 工程技术-工程:工业
CiteScore
23.30
自引率
13.20%
发文量
216
审稿时长
25 days
期刊介绍: The Journal of Manufacturing Systems is dedicated to showcasing cutting-edge fundamental and applied research in manufacturing at the systems level. Encompassing products, equipment, people, information, control, and support functions, manufacturing systems play a pivotal role in the economical and competitive development, production, delivery, and total lifecycle of products, meeting market and societal needs. With a commitment to publishing archival scholarly literature, the journal strives to advance the state of the art in manufacturing systems and foster innovation in crafting efficient, robust, and sustainable manufacturing systems. The focus extends from equipment-level considerations to the broader scope of the extended enterprise. The Journal welcomes research addressing challenges across various scales, including nano, micro, and macro-scale manufacturing, and spanning diverse sectors such as aerospace, automotive, energy, and medical device manufacturing.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信