Generate Descriptions of Medical Dialogues Through Two-Layers Transformer-Based Summarization

Abstract

This paper presents a two-layer transformer-based summarization approach for generating descriptions of medical dialogues, with a focus on the concept that using small amounts of training data can produce promising results and avoid the need for high computing resources. Previous research has primarily utilized the full or most of the HealthCareMagic dataset, comprising 225,872 medical dialogues for training, while this study uses much smaller training datasets of 5000 and 7000 instances. Despite the smaller training sets, our approach yields promising results, with a challenging improvement of ROUGE evaluation metric up to 0.44 compared to previous work. The two-layer summarization technique involves the BART pre-trained transformer model for generating medical dialogue summaries, followed by fine-tuning the T5 pre-trained model to generate one-statement descriptions for each summary. This research has practical implications for improving medical dialogue summarization, enabling healthcare professionals to review and assess patient information more quickly.