A neural network to identify requests, decisions, and arguments in court rulings on custody

Court rulings are among the most important documents in all legal systems. This article describes a study in which natural language processing is used for the automatic characterization of Spanish judgments that deal with the physical custody (joint or individual) of minors. The model was trained to identify a set of elements: the type of custody requested by the plaintiff, the type of custody decided on by the court, and eight of the most commonly used arguments in this type of judgment. Two jurists independently annotated more than 3000 judgments, which were used to train a model based on transformers. The main difficulties encountered in this task were the complexity of the judicial language and the need to work with appellate court rulings that have a more complicated structure than decisions at first instance. For the complete court rulings, the F1 score of the inter-annotator agreement ranged from 0.60 to 0.86 and the Kappa index from 0.33 to 0.73. The F1 score of the agreement between the model and the annotators ranged from 0.66 to 0.93 and the Kappa index from 0.57 to 0.80. These results in which the model performance exceeds even the inter-annotator agreement show the high ability of transformers to identify abstract entities in legal texts.