基于多模态表征学习的以任务为中心的工业维护自动化知识图谱构建方法

Zengkun Liu, Yuqian Lu
{"title":"基于多模态表征学习的以任务为中心的工业维护自动化知识图谱构建方法","authors":"Zengkun Liu, Yuqian Lu","doi":"10.1002/eng2.12952","DOIUrl":null,"url":null,"abstract":"Maintenance manuals are crucial information sources for maintenance and repair. Prior studies explored factual knowledge extraction from textual documents. However, maintenance knowledge in manuals is more task‐centric rather than factual knowledge and often documented in an unstructured Portable Document Format (PDF), posing challenges for knowledge extraction. Addressing this, this research develops effective methods to extract task‐centric maintenance knowledge from unstructured PDF manuals. A new Task‐centric Knowledge Graph (TCKG) schema centralized on maintenance task components (MTCs) is proposed to address the need for structured knowledge representation. A method (Heterogeneous Graph‐based Method, HGM) for knowledge extraction is then proposed, which is enhanced by incorporating visual and spatial information. In the experiments, the proposed HGM exhibits robust performance in the knowledge extraction process, surpassing the baseline Graph‐based Interaction Model with a Tracker (GIT) method in MTCs extraction by 13.3%, and the baseline Translate Embedding (TransE) method in MTCs' relation extraction by 3.8%. A series of ablation studies also prove that including visual and spatial information through the proposed method can improve the relation extraction performance by over 10%. This research supplies valuable insights for future developments in information extraction from maintenance manuals.","PeriodicalId":502604,"journal":{"name":"Engineering Reports","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation\",\"authors\":\"Zengkun Liu, Yuqian Lu\",\"doi\":\"10.1002/eng2.12952\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Maintenance manuals are crucial information sources for maintenance and repair. Prior studies explored factual knowledge extraction from textual documents. However, maintenance knowledge in manuals is more task‐centric rather than factual knowledge and often documented in an unstructured Portable Document Format (PDF), posing challenges for knowledge extraction. Addressing this, this research develops effective methods to extract task‐centric maintenance knowledge from unstructured PDF manuals. A new Task‐centric Knowledge Graph (TCKG) schema centralized on maintenance task components (MTCs) is proposed to address the need for structured knowledge representation. A method (Heterogeneous Graph‐based Method, HGM) for knowledge extraction is then proposed, which is enhanced by incorporating visual and spatial information. In the experiments, the proposed HGM exhibits robust performance in the knowledge extraction process, surpassing the baseline Graph‐based Interaction Model with a Tracker (GIT) method in MTCs extraction by 13.3%, and the baseline Translate Embedding (TransE) method in MTCs' relation extraction by 3.8%. A series of ablation studies also prove that including visual and spatial information through the proposed method can improve the relation extraction performance by over 10%. This research supplies valuable insights for future developments in information extraction from maintenance manuals.\",\"PeriodicalId\":502604,\"journal\":{\"name\":\"Engineering Reports\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-07-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Engineering Reports\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1002/eng2.12952\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Engineering Reports","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1002/eng2.12952","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

维护手册是维护和维修的重要信息来源。先前的研究探讨了从文本文档中提取事实知识的问题。然而,手册中的维护知识更多的是以任务为中心,而不是事实知识,而且通常以非结构化的便携式文档格式(PDF)记录,这给知识提取带来了挑战。为此,本研究开发了从非结构化 PDF 手册中提取以任务为中心的维护知识的有效方法。为满足结构化知识表示的需要,提出了一种以维护任务组件(MTC)为中心的新的以任务为中心的知识图谱(TCKG)模式。然后提出了一种知识提取方法(基于异构图的方法,HGM),该方法通过结合视觉和空间信息得到了增强。在实验中,所提出的 HGM 在知识提取过程中表现出强劲的性能,在 MTCs 提取方面比基准的基于图形的跟踪器交互模型(GIT)方法高出 13.3%,在 MTCs 关系提取方面比基准的翻译嵌入(TransE)方法高出 3.8%。一系列消融研究还证明,通过建议的方法纳入视觉和空间信息,可以将关系提取性能提高 10%以上。这项研究为维护手册信息提取的未来发展提供了宝贵的启示。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation
Maintenance manuals are crucial information sources for maintenance and repair. Prior studies explored factual knowledge extraction from textual documents. However, maintenance knowledge in manuals is more task‐centric rather than factual knowledge and often documented in an unstructured Portable Document Format (PDF), posing challenges for knowledge extraction. Addressing this, this research develops effective methods to extract task‐centric maintenance knowledge from unstructured PDF manuals. A new Task‐centric Knowledge Graph (TCKG) schema centralized on maintenance task components (MTCs) is proposed to address the need for structured knowledge representation. A method (Heterogeneous Graph‐based Method, HGM) for knowledge extraction is then proposed, which is enhanced by incorporating visual and spatial information. In the experiments, the proposed HGM exhibits robust performance in the knowledge extraction process, surpassing the baseline Graph‐based Interaction Model with a Tracker (GIT) method in MTCs extraction by 13.3%, and the baseline Translate Embedding (TransE) method in MTCs' relation extraction by 3.8%. A series of ablation studies also prove that including visual and spatial information through the proposed method can improve the relation extraction performance by over 10%. This research supplies valuable insights for future developments in information extraction from maintenance manuals.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信