Order in Chaos: Prioritizing Mobile App Reviews using Consensus Algorithms

Abstract

The continuous growth of the mobile apps industry creates a competition among apps developers. To succeed, app developers must attract and retain users. User reviews provide a wealth of information about bugs to fix and features to add and can help app developers offer high-quality apps. However, apps may receive hundreds of unstructured reviews, which makes transforming them into change requests a difficult task. Approaches exist for analyzing and extracting topics from mobile app reviews, however, prioritizing these reviews has not gained much attention. In this study, we introduce the use of a consensus algorithm to help developers prioritize user reviews for the purpose of app evolution. We evaluate the usefulness of our approach and meaningfulness of its consensus rankings on four Android apps. We compare the rankings against reviews ranked by app developers manually and show that there is a strong correlation between the two (average Kendall rank correlation coefficient = 0.516). Thus, our approach can prioritize user reviews and help developers focus their time/effort on improving their apps instead of on identifying reviews to address in the next release.