IterSum: Iterative summarization based on document topological structure

Abstract

Document structure plays a crucial role in understanding and analyzing document information. However, effectively encoding document structural features into the Transformer architecture faces significant challenges. This is primarily because different types of documents require the model to adopt varying structural encoding strategies, leading to a lack of a unified framework that can broadly adapt to different document types to leverage their structural properties. Despite the diversity of document types, sentences within a document are interconnected through semantic relationships, forming a topological semantic network. This topological structure is essential for integrating and summarizing information within the document. In this work, we introduce IterSum, a versatile text summarization framework applicable to various types of text. In IterSum, we utilize the document’s topological structure to divide the text into multiple blocks, first generating a summary for the initial block, then combining the current summary with the content of the next block to produce the subsequent summary, and continuing in this iterative manner until the final summary is generated. We validated our model on nine different types of public datasets, including news, knowledge bases, legal documents, and guidelines. Both quantitative and qualitative analyses were conducted, and the experimental results show that our model achieves state-of-the-art performance on all nine datasets measured by ROUGE scores. We also explored low-resource summarization, finding that even with only 10 or 100 samples in multiple datasets, top-notch results were obtained. Finally, we conducted human evaluations to further validate the superiority of our model.