Adopting machine translation in the healthcare sector: A methodological multi-criteria review

Abstract

Background:

The recent advances in machine translation (MT) offer an appealing and low-cost solution to overcome language barriers in multiple contexts (e.g., travelling, cultural interaction, digital content localisation). However, highly-technical domains typically exhibiting as long, complex, and specialised texts as the healthcare sector, pose multiple challenges to the effective and risk-safe use of MT.

Methods:

To examine how MT nowadays assists written/verbal health communication and because of the existing considerable heterogeneity in technological enablers, language pairs and user groups, training approaches, evaluation processes, and users” requirements, we propose in this paper a methodological multi-criteria literature review based on current guidelines in computer science research and grounded on a customised configuration of the PRISMA methodology, normally used to perform meta-analyses on clinical trials. The review focuses on language-to-language medical MT, covers the time period January 2015–February 2023, and only refers to articles written in English that are accessible via four scientific online digital libraries. Articles are ranked according to a meta-evaluation scoring method for MT scientific credibility along with a scoring for assessing the scope of MT in healthcare. Finally, a guideline to properly design a study about MT in healthcare is also proposed.

Results:

The review included a final set of 58 articles from journals ($n=30$) and conference proceedings ($n=28$), considering 48 different language combinations. We identified a predominance of English-to-Spanish ($n=19$) and English-to-Chinese ($n=16$) implementations, mainly tailored to medical staff only ($n=14$) or along with patients ($n=12$). Included papers addressed clinical communication ($n=21$) and health education ($n=37$). Unidirectional real-time bilingual MT ($n=24$) was the most frequent configuration. MT implementations were dominated by Google Translate ($n=22$) often used as baseline, OpenNMT ($n=12$), or Moses ($n=11$). Training and evaluation approaches varied considerably, while deployment and pre-/post-editing were rarely described with an adequate level of detail.

Conclusion:

Even if a significant number of articles reported that the proposed MT solutions were effective when translating (bio)medical texts, only a subset of them complied with rigorous translation quality assessment criteria (e.g., use of automatic metrics better related to human ranking than BLEU or statistical significance testing). Nevertheless, MT can be a valid support/supplement in health communication but to cope with issues in fluency, accuracy, unnatural translations, domain-adequacy, and potential safety risks (for highly-sensitive documents), appropriate MT training is essential, along with in-domain human post-editing. The presence of in-domain training text corpora has also proven to be beneficial. Finally, guidelines about how to design studies on MT in healthcare are also proposed to engage more researchers in this field.