Automatic Summarization of Scientific Documents Using Transformer Architectures: A Review

Abstract

As technology advances, the volume of textual material produced on the web has been steadily rising. It can take a lot of time and effort to extract useful information from textual data. Automatic text summarizing aims to create concise summaries that retain the most important parts of the source document. Transformer-based architectures have demonstrated excellently in Natural Language Processing (NLP), particularly when it comes to summarizing textual content. This paper presents a thorough analysis of the most recent advancements in transformer topologies for automatic text summarization, with a focus on the Bidirectional Autoregressive Transformer (BART). This paper highlights future directions for research on transformer-like models for autonomous text summarization, such as BART and BERT.