Automated Outlier Removal for Mobile Microbenchmarking Datasets

Abstract

Microbenchmarking is a useful tool for fine-grained performance analysis, and represents a potentially valuable tool in the development of mobile applications and systems. However, the fine-grained measurements of microbenchmarking are inherently susceptible to noise from the underlying operating system and hardware. This noise includes outliers that must be removed in order to produce meaningful results. Existing microbenchmarking implementations utilise only simple mechanisms for removing outliers. In this paper we propose a heuristic for the automated removal of outliers from mobile microbenchmarking datasets. We then simplify this heuristic for use on mobile devices. Empirical evaluation demonstrates that our outlier removal heuristics are effective across microbenchmarking datasets collected from a range of mobile devices. Our simplified heuristic operates in log-linear time, making it suitable for use on resource-constrained mobile devices. The ability to perform outlier removal on-device without the need for post-processing on desktop or server hardware enhances the utility of mobile microbenchmarking tools. Our results present interesting opportunities for further studies across a broader range of device platforms.