Abstractive Dialog Summarization using Two Stage Framework with Contrastive Learning

Abstract

In the modern era, a large amount of text conversation data between two or more interlocutors is generated by different online service consumers every hour. Converting such a long conversation into a concise form is more useful for further analysis and can boost service quality when conducted in an efficient manner. Abstractive summarization models usually suffer from performance degradation due to the different objective functions used in the training and inference steps. Contrastive learning is a powerful technique for developing training objectives that are similar to evaluation metrics and thus improve performance. Two-stage framework with contrastive learning are gaining popularity to mitigate this gap but this approach is very daunting in the field because of its huge computation time and demand for memory usage. To address this issue, we propose an optimization in the two-stage framework architecture for dialog summarization using the ALBERT pre-trained model in the evaluator section which is more efficient with respect to the usage of resources. Our method significantly outperforms strong baselines on SAMSum and DialogSum dataset for abstractive dialog summarization task.