Estimating Quality Ratings from Touch Interactions in Mobile Games

Abstract

Obtaining reliable quality ratings for applications is a time-consuming and expensive process. Touch interaction data, however, can be recorded without additional effort when using mobile applications. It could be used to assess usability without having to conduct dedicated tests. This work investigates the possibility to estimate quality ratings for two mobile games solely from touch interactions without relying on performance or game logic measures. To capture pragmatic as well as hedonic quality aspects of usability as ground truth, the AttrakDiff Mini questionnaire was chosen. Two different simple mobile games were used to capture a variety of touch interactions (taps and swipes). To influence the user ratings, three versions of the games were presented: one in original quality, the two others with manipulated interaction quality. Touch interaction features were extracted from the recorded interaction data and used for training models using linear regression and multivariate adaptive regression splines (MARS). The models were built with three times 10-fold cross-validation on each game data set separately. The results indicate that the ratings for pragmatic quality are estimated significantly better than for hedonic quality and attractiveness. Furthermore, the transferability of the models from one game to the other was examined and the touch features that were most important for estimating quality ratings in this study were identified. Touch interactions seem to carry some information on usability, especially pragmatic quality, and it seems to be possible to estimate a broad direction of the user's usability perception. Just a few touch features carry the most information and can be used for simple but fast and powerful models. This could be applied as an automatic analytic tool for mobile applications as well as in adaptive applications that optimize themselves in terms of usability perceived by individual users.