Mobile application review summarization using chain of density prompting

Abstract

Mobile app users commonly rely on app store ratings and reviews to find apps that suit their needs. However, the sheer volume of reviews available on app stores can lead to information overload, thus impeding users’ ability to make informed app selection decisions. To overcome this limitation, in this paper, we leverage Large Language Models (LLMs) to summarize mobile app reviews. In particular, we use the Chain of Density (CoD) prompt to guide OpenAI GPT-4 to generate abstractive, semantically dense, and readable summaries of mobile app reviews. The CoD prompt is engineered to iteratively extract salient entities from the source text and fuse them into a fixed-length summary. We evaluate the performance of our approach using a large dataset of mobile app reviews. We further conduct an empirical evaluation with 48 study participants to assess the readability of the generated CoD summaries. Our results show that an altered CoD prompt can correctly identify the main themes in user reviews and consolidate them into a natural language summary that is intended for end-user consumption. The prompt also manages to maintain the readability of the generated summaries while increasing their density. Our work in this paper aims to substantially improve mobile app users’ experience by providing an effective mechanism for summarizing important user feedback in the review stream.