A semantic enhancement-based multimodal network model for extracting information from evidence lists

Abstract

Courts require the extraction of crucial information about various cases from heterogeneous evidence lists for knowledge-driven decision-making. However, traditional manual screening is complex and inaccurate when confronted with massive evidence lists and cannot meet the demands of legal judgment. Therefore, we propose a semantic enhancement-based multimodal network model (SEBM) to accurately extract critical information from evidence lists. First, we construct the entity semantic graph based on the differences among entity categories in the text content. Subsequently, we extract the features of multiple modalities within the document by employing distinct methods and guide the fusion of features within each modality to enhance the semantic association among them based on the constructed entity semantic graphs. Furthermore, the improved multimodal self-attention mechanism is employed to enhance the interactions between the various modal features, and the loss function combining Taylor polynomials and supervised contrast learning is utilized to reduce the information loss. Finally, SEBM is evaluated using the authentic Chinese evidence list dataset, which includes extensive entity details from diverse case types across multiple law firms. Results from experiments conducted on the authentic evidence list dataset demonstrate that our model performs better than other high-performing models.