Speech translation for multilingual medical education leveraged by large language models

Abstract

The application of large language models (LLMs) to speech translation (ST) or, in general, to machine translation (MT) has recently provided excellent results, superseding conventional encoder–decoder MT systems in the general domain. However, this is not clearly the case when LLMs as MT systems are translating medical-related materials. In this respect, the provision of multilingual training materials for oncology professionals is a goal of the EU project Interact-Europe in which this work was framed. To this end, cross-language technology adapted to the oncology domain was developed, evaluated and deployed for multilingual interspecialty medical education. More precisely, automatic speech recognition (ASR) and MT models were adapted to the oncology domain to translate English pre-recorded training videos, kindly provided by the European School of Oncology (ESO), into French, Spanish, German and Slovene. In this work, three categories of MT models adapted to the medical domain were assessed: bilingual encoder–decoder MT models trained from scratch, pre-trained large multilingual encoder–decoder MT models, and multilingual decoder-only LLMs. The experimental results underline the competitiveness in translation quality of LLMs compared to encoder–decoder MT models. Finally, the ESO speech dataset, comprising roughly 1000 videos and 745 h for the training and evaluation of ASR, MT and ST models, was publicly released for the scientific community.