{"title":"如果你可以,就冻结它:智能手机性能基准的挑战和未来方向","authors":"Yao Guo, Yunnan Xu, Xiangqun Chen","doi":"10.1145/3032970.3032979","DOIUrl":null,"url":null,"abstract":"Benchmarking the performance of mobile devices such as Android-based smartphones is important in understanding and comparing the performance of different devices. Performance benchmarking tools such as Antutu have been widely used in both academia and industry. However, one of the main difficulties when benchmarking smartphone performance is due to the fact that the performance cannot be measured accurately and steadily. This paper investigates the challenges on performance benchmarking for Android-based smartphones. We identify key factors affecting performance benchmarking, which include frequency scaling setting, temperature, running background services, etc. Experiments show that some of these factors may cause performance fluctuation by as high as 60%. We show preliminary results in controlling the performance benchmarking process to generate steady results, for example freezing a smartphone in a refrigerator could remove most of the fluctuations. Finally, we discuss the implications of our study and possible future research directions.","PeriodicalId":309322,"journal":{"name":"Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications","volume":"1 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2017-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"6","resultStr":"{\"title\":\"Freeze It If You Can: Challenges and Future Directions in Benchmarking Smartphone Performance\",\"authors\":\"Yao Guo, Yunnan Xu, Xiangqun Chen\",\"doi\":\"10.1145/3032970.3032979\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Benchmarking the performance of mobile devices such as Android-based smartphones is important in understanding and comparing the performance of different devices. Performance benchmarking tools such as Antutu have been widely used in both academia and industry. However, one of the main difficulties when benchmarking smartphone performance is due to the fact that the performance cannot be measured accurately and steadily. This paper investigates the challenges on performance benchmarking for Android-based smartphones. We identify key factors affecting performance benchmarking, which include frequency scaling setting, temperature, running background services, etc. Experiments show that some of these factors may cause performance fluctuation by as high as 60%. We show preliminary results in controlling the performance benchmarking process to generate steady results, for example freezing a smartphone in a refrigerator could remove most of the fluctuations. Finally, we discuss the implications of our study and possible future research directions.\",\"PeriodicalId\":309322,\"journal\":{\"name\":\"Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications\",\"volume\":\"1 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2017-02-21\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"6\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3032970.3032979\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3032970.3032979","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Freeze It If You Can: Challenges and Future Directions in Benchmarking Smartphone Performance
Benchmarking the performance of mobile devices such as Android-based smartphones is important in understanding and comparing the performance of different devices. Performance benchmarking tools such as Antutu have been widely used in both academia and industry. However, one of the main difficulties when benchmarking smartphone performance is due to the fact that the performance cannot be measured accurately and steadily. This paper investigates the challenges on performance benchmarking for Android-based smartphones. We identify key factors affecting performance benchmarking, which include frequency scaling setting, temperature, running background services, etc. Experiments show that some of these factors may cause performance fluctuation by as high as 60%. We show preliminary results in controlling the performance benchmarking process to generate steady results, for example freezing a smartphone in a refrigerator could remove most of the fluctuations. Finally, we discuss the implications of our study and possible future research directions.