{"title":"同意不同意:关于标记有用的应用程序评论","authors":"Andrew J. Simmons, Leonard Hoon","doi":"10.1145/3010915.3010976","DOIUrl":null,"url":null,"abstract":"Mobile apps designers seek to prioritise and refine app features so as to optimise user experience across the ensemble of possible situations and contexts in which the app is used. App reviews---some helpful, others irrelevant---can be analysed for feedback on this user experience. However, few studies have specifically examined the helpfulness of app reviews. In this paper, we surveyed users and developers to rate 167 reviews for helpfulness, obtaining a total of 2,558 helpfulness ratings captured on a 5 point Likert scale. We found only slight agreement (nominal Krippendorff's alpha = 0.039) between participants on the helpfulness of reviews. Differences between reviews become evident when we summarise all the helpfulness ratings per review. We conclude that the disagreement among users limits the potential of mobile app review recommender systems.","PeriodicalId":309823,"journal":{"name":"Proceedings of the 28th Australian Conference on Computer-Human Interaction","volume":"102 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2016-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"8","resultStr":"{\"title\":\"Agree to disagree: on labelling helpful app reviews\",\"authors\":\"Andrew J. Simmons, Leonard Hoon\",\"doi\":\"10.1145/3010915.3010976\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Mobile apps designers seek to prioritise and refine app features so as to optimise user experience across the ensemble of possible situations and contexts in which the app is used. App reviews---some helpful, others irrelevant---can be analysed for feedback on this user experience. However, few studies have specifically examined the helpfulness of app reviews. In this paper, we surveyed users and developers to rate 167 reviews for helpfulness, obtaining a total of 2,558 helpfulness ratings captured on a 5 point Likert scale. We found only slight agreement (nominal Krippendorff's alpha = 0.039) between participants on the helpfulness of reviews. Differences between reviews become evident when we summarise all the helpfulness ratings per review. We conclude that the disagreement among users limits the potential of mobile app review recommender systems.\",\"PeriodicalId\":309823,\"journal\":{\"name\":\"Proceedings of the 28th Australian Conference on Computer-Human Interaction\",\"volume\":\"102 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2016-11-29\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"8\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the 28th Australian Conference on Computer-Human Interaction\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3010915.3010976\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 28th Australian Conference on Computer-Human Interaction","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3010915.3010976","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Agree to disagree: on labelling helpful app reviews
Mobile apps designers seek to prioritise and refine app features so as to optimise user experience across the ensemble of possible situations and contexts in which the app is used. App reviews---some helpful, others irrelevant---can be analysed for feedback on this user experience. However, few studies have specifically examined the helpfulness of app reviews. In this paper, we surveyed users and developers to rate 167 reviews for helpfulness, obtaining a total of 2,558 helpfulness ratings captured on a 5 point Likert scale. We found only slight agreement (nominal Krippendorff's alpha = 0.039) between participants on the helpfulness of reviews. Differences between reviews become evident when we summarise all the helpfulness ratings per review. We conclude that the disagreement among users limits the potential of mobile app review recommender systems.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
