Keyword Generation for Russian-Language Scientific Texts Using the mT5 Model

Abstract

The authors propose an approach to generate keywords for Russian-language scientific texts using the mT5 (multilingual text-to-text transformer) model, fine-tuned on the Keyphrases CS&Math Russian text corpus. Automatic keyword selection is an urgent task in natural language processing, since keywords help readers search for articles and facilitate the systematization of scientific texts. In this paper, the task of selecting keywords is considered as a task of automatic text abstracting. Additional training of mT5 is carried out on the texts of abstracts of Russian-language scientific articles. The input and output data are abstracts and comma-separated lists of keywords, respectively. The results obtained using mT5 are compared with the results of several basic methods: TopicRank, YAKE!, RuTermExtract, and KeyBERT. The following metrics are used to present the results: F‑measure, ROUGE-1, and BERTScore. The best results on the test sample are obtained using mT5 and RuTermExtract. The highest F-measure is demonstrated by the mT5 model (11.24%), surpassing RuTermExtract by 0.22%. RuTermExtract shows the best result according to the ROUGE-1 metric (15.12%). The best results for BERTScore are also achieved by these two methods: mT5, 76.89% (BERTScore using the mBERT model); RuTermExtract, 75.8% (BERTScore based on ruSciBERT). The authors also assess the ability of mT5 to generate keywords that are not in the source text. The limitations of the proposed approach include the need to form a training sample for additional model training and probably the limited applicability of the additional trained model for texts in other subject areas. The advantages of keyword generation using mT5 are the absence of the need to set fixed values for the length and number of keywords, the need for normalization, which is especially important for inflected languages, and the ability to generate keywords that are not explicitly present in the text.