A Chinese Multi-modal Relation Extraction Model for Internet Security of Finance

Abstract

As the base of the whole economy and society, internet security of finance directly affects the overall development of the country. With the development of the Internet, it is essential to effectively extract the relation between financial entities from internet financial intelligence and build a financial security knowledge graph, which lays the foundation for monitoring of internet security of finance. For relation extraction of Chinese internet financial intelligence, the existing models are all based on single-modal text semantics ignoring the role of Chinese pictographic semantics, while the shape and structure of Chinese characters contains useful semantics. In addition, the pictographic semantic fusion method of Chinese text also needs to be improved for better performance. To solve these shortcomings, we propose a Chinese Multimodal Relation Extraction model (CMRE), which improves the relation extraction ability on the Chinese internet financial intelligence. In CMRE, we extract pictographic semantics based on Chinese character shape and structure. Furthermore, we design a novel multi-modal semantic fusion module based on improved Transformer to effectively fuse the text and pictographic semantics. Additionally, we design experiments on the Chinese literature dataset(Sanwen) to test the relation extraction capability of CMRE. Finally, we employ CMRE to extract relations between financial entities on the internet financial intelligence dataset(FinRE) to compare with other baseline models.