Demystifying “removed reviews” in iOS app store

软件产业与工程 Pub Date : 2022-11-07 DOI:10.1145/3540250.3558966

Liu Wang, Haoyu Wang, Xiapu Luo, Tao Zhang, Shangguang Wang, Xuanzhe Liu

{"title":"Demystifying “removed reviews” in iOS app store","authors":"Liu Wang, Haoyu Wang, Xiapu Luo, Tao Zhang, Shangguang Wang, Xuanzhe Liu","doi":"10.1145/3540250.3558966","DOIUrl":null,"url":null,"abstract":"The app markets enable users to submit feedback for downloaded apps in the form of star ratings and text reviews, which are meant to be helpful and trustworthy for decision making to both developers and other users. App markets have released strict guidelines/policies for user review submissions. However, there has been growing evidence showing the untrustworthy and poor-quality of app reviews, making the app store review environment a shambles. Therefore, review removal is a common practice, and market maintainers have to remove undesired reviews from the market periodically in a reactive manner. Although some reports and news outlets have mentioned removed reviews, our research community still lacks the comprehensive understanding of the landscape of this kind of reviews. To fill the void, in this paper, we present a large-scale and longitudinal study of removed reviews in iOS App Store. We first collaborate with our industry partner to collect over 30 million removed reviews for 33,665 popular apps over the course of a full year in 2020. This comprehensive dataset enables us to characterize the overall landscape of removed reviews. We next investigate the practical reasons leading to the removal of policy-violating reviews, and summarize several interesting reasons, including fake reviews, offensive reviews, etc. More importantly, most of these mis-behaviors can be reflected on reviews’ basic information including the posters, narrative content, and posting time. It motivates us to design an automated approach to flag the policy-violation reviews, and our experiment result on the labelled benchmark can achieve a good performance (F1=97%). We further make an attempt to apply our approach to the large-scale industry setting, and the result suggests the promising industry usage scenario of our approach. Our approach can act as a gatekeeper to pinpoint policy-violation reviews beforehand, which will be quite effective in improving the maintenance process of app reviews in the industrial setting.","PeriodicalId":68155,"journal":{"name":"软件产业与工程","volume":"219 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2022-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"软件产业与工程","FirstCategoryId":"1089","ListUrlMain":"https://doi.org/10.1145/3540250.3558966","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 2

Abstract

The app markets enable users to submit feedback for downloaded apps in the form of star ratings and text reviews, which are meant to be helpful and trustworthy for decision making to both developers and other users. App markets have released strict guidelines/policies for user review submissions. However, there has been growing evidence showing the untrustworthy and poor-quality of app reviews, making the app store review environment a shambles. Therefore, review removal is a common practice, and market maintainers have to remove undesired reviews from the market periodically in a reactive manner. Although some reports and news outlets have mentioned removed reviews, our research community still lacks the comprehensive understanding of the landscape of this kind of reviews. To fill the void, in this paper, we present a large-scale and longitudinal study of removed reviews in iOS App Store. We first collaborate with our industry partner to collect over 30 million removed reviews for 33,665 popular apps over the course of a full year in 2020. This comprehensive dataset enables us to characterize the overall landscape of removed reviews. We next investigate the practical reasons leading to the removal of policy-violating reviews, and summarize several interesting reasons, including fake reviews, offensive reviews, etc. More importantly, most of these mis-behaviors can be reflected on reviews’ basic information including the posters, narrative content, and posting time. It motivates us to design an automated approach to flag the policy-violation reviews, and our experiment result on the labelled benchmark can achieve a good performance (F1=97%). We further make an attempt to apply our approach to the large-scale industry setting, and the result suggests the promising industry usage scenario of our approach. Our approach can act as a gatekeeper to pinpoint policy-violation reviews beforehand, which will be quite effective in improving the maintenance process of app reviews in the industrial setting.

查看原文本刊更多论文

破解iOS应用商店中的“删除评论

应用市场允许用户以星级和文字评论的形式提交下载应用的反馈，这对开发者和其他用户的决策都是有帮助和值得信赖的。应用市场针对用户评论发布了严格的指导方针/政策。然而，越来越多的证据表明应用评论不可信且质量低劣，这使得应用商店的评论环境变得一团糟。因此，删除评论是一种常见的做法，市场维护者必须以一种被动的方式定期地从市场中删除不需要的评论。虽然有一些报道和新闻媒体提到了删除评论，但我们的研究界仍然缺乏对这类评论的全面了解。为了填补这一空白，我们在本文中对iOS App Store中被删除的评论进行了大规模的纵向研究。我们首先与我们的行业合作伙伴合作，在2020年的一整年里，为33,665个流行应用收集了超过3000万条被删除的评论。这个全面的数据集使我们能够描述被删除评论的整体情况。接下来，我们调查了导致删除违反政策评论的实际原因，并总结了几个有趣的原因，包括虚假评论、冒犯性评论等。更重要的是，这些错误行为大多可以反映在评论的基本信息上，包括海报、叙述内容和发布时间。这促使我们设计一种自动化的方法来标记策略违规审查，我们在标记基准上的实验结果可以达到很好的性能(F1=97%)。我们进一步尝试将我们的方法应用到大型工业环境中，结果表明我们的方法具有良好的工业应用前景。我们的方法可以起到看门人的作用，提前找出违反政策的评论，这将非常有效地改善工业环境中应用评论的维护过程。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

软件产业与工程

自引率

0.00%

发文量

676