More attention for computer-aided conceptual design: A multimodal data-driven interactive design method

Computer-aided conceptual design (CACD) is a core means for the development of new products, as it can materialize designers’ inherent thinking. However, when designers encounter stagnation during CACD, they need to consult third-party design knowledge to seek inspiration, which frequently disrupts their design thinking process. Deep learning-empowered design methods and design knowledge management can be a potential solution to address these issues. This study proposes a multimodal design data-driven interactive design method. Multimodal data are utilized to identify the designer’s implicit intentions while design attention is abstracted to match relevant knowledge as computer feedback. It achieves the “designer-computer-designer” closed-loop interactive design through the mediation of design attention. The multimodal design data (design images and design descriptions) is obtained through sketch modeling and verbal protocol analysis experiments. A multimodal Transformer based on T2T-ViT and Bert (TB-Multiformer) is constructed to capture multimodal features to identify conceptual design intentions by utilizing cross-modal design attention modules and self-design attention modules. Since the identified attention can be used to match the knowledge that designers are more concerned about, an attention-based design knowledge recommendation method (AbDKR) is proposed to provide proactive knowledge feedback. It can prevent designers from spending time searching for design knowledge and helps them maintain sufficient inspiration. A case study on the conceptual design of two types of mechanical structure is conducted to illustrate the feasibility and practicability of the proposed approach.