CSKINet: A multimodal network model integrating conceptual semantic knowledge injection for relation extraction of Chinese corporate reports

Recognizing the associations among entities in corporate reports accurately is crucial for market regulation and policy development. Nevertheless, confronted with massive corporate information, the traditional manual screening approach is cumbersome, struggling to match the demand. Consequently, we propose a multimodal network model incorporating conceptual semantic knowledge injection, CSKINet, for accurately extracting relations from Chinese corporate reports. The essential highlights in the design of the CSKINet model are the following: (1) Integrate the conceptual descriptions of corporations from external resources to construct the semantic knowledge repository of corporate concepts, which provides a solid semantic foundation for the model. (2) Multimodal features are extracted from the documents by various means and corporate conceptual knowledge is integrated into the model representation to enhance the representation capability of the model. (3) The multimodal self-attention mechanism that captures cross-modal associations and the biaffine classifier with Taylor polynomial loss function that optimizes training iterations further improve the learning efficiency and prediction accuracy. The results on the real corporate report dataset show that our proposed model can more accurately extract the relations from Chinese corporate reports compared to other baseline models, where the F1 score reaches 85.76%.