A question answering system for assembly process of wind turbines based on multi-modal knowledge graph and large language model

AbstractIn the field of wind power generation, wind turbines serve as the foundation for harnessing electrical energy. However, the assembly process information for wind turbines is typically dispersed among various modalities such as 3D models, natural text, and images in the form of process documents. The difficulty in effectively utilising historical process knowledge hampers the efficiency of assembly process design and subsequently affects production efficiency. To address this issue, this paper constructs a Multi-modal Process Knowledge Graph for Wind Turbines, named MPKG-WT. Additionally, a wind turbine assembly process question-answering system combining multi-modal knowledge graphs with large language models (LLMs) is proposed to enable efficient utilisation of historical assembly process knowledge. The proposed approach achieves outstanding results when compared with other state-of-the-art KBQA methods and recent LLMs using a wind turbine assembly process dataset. The effectiveness of the approach is further validated through a visualised assembly process question-answering system. The research findings demonstrate a significant improvement in assembly process design efficiency.KEYWORDS: Multi-modal knowledge graphWind turbinesAssembly process knowledgeLarge language modelQuestion answering Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationFundingThis work was supported by National Key Research and Development Program of China: [grant no 2019YFB1706300]; Shanghai Rising-Star Plan (Yangfan Program) from the Science and Technology Commission of Shanghai Municipality: [grant no 22YF1400200].