Experimental Performance Evaluation Among Cloud Infrastructure Providers Under Different Load Levels

2021 XLVII Latin American Computing Conference (CLEI) Pub Date : 2021-10-25 DOI:10.1109/CLEI53233.2021.9639943

Denis B. Oliveira, R. R. Oliveira, R. F. Vilela, V. H. S. C. Pinto, Roberto N. Ungarelli

{"title":"Experimental Performance Evaluation Among Cloud Infrastructure Providers Under Different Load Levels","authors":"Denis B. Oliveira, R. R. Oliveira, R. F. Vilela, V. H. S. C. Pinto, Roberto N. Ungarelli","doi":"10.1109/CLEI53233.2021.9639943","DOIUrl":null,"url":null,"abstract":"Background: Performance testing can estimate the capacity of a web service under requests. Decision-making on the ideal cloud service infrastructures for deploying specific cloud applications is challenging. Goal: Investigation on the performance of cloud infrastructure providers under different load levels. Method: An experimental study evaluated Amazon, Azure, Google, and IBM cloud infrastructure providers in terms of performance under Infrastructure as a Service (IaaS) perspective. Results: The results indicated satisfactory performance in response time and latency among most providers when subjected to up to 300 simultaneous threads. However, this effect decreases as the number of threads increases, and Apdex index value and level of user's satisfaction are significantly reduced under different load levels, mainly for the IBM provider. Besides, the error rate rises substantially at 400 threads, with more critical results for IBM and Amazon providers. Conclusions: Providers show distinct differences across metrics, and the data collected during testings reinforced the potential particularities of cloud infrastructure services for the choice of a provider. Although preliminary, such results can support software companies in selecting a proper infrastructure provider according to particular requirements.","PeriodicalId":6803,"journal":{"name":"2021 XLVII Latin American Computing Conference (CLEI)","volume":"70 1","pages":"1-10"},"PeriodicalIF":0.0000,"publicationDate":"2021-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2021 XLVII Latin American Computing Conference (CLEI)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/CLEI53233.2021.9639943","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 1

Abstract

Background: Performance testing can estimate the capacity of a web service under requests. Decision-making on the ideal cloud service infrastructures for deploying specific cloud applications is challenging. Goal: Investigation on the performance of cloud infrastructure providers under different load levels. Method: An experimental study evaluated Amazon, Azure, Google, and IBM cloud infrastructure providers in terms of performance under Infrastructure as a Service (IaaS) perspective. Results: The results indicated satisfactory performance in response time and latency among most providers when subjected to up to 300 simultaneous threads. However, this effect decreases as the number of threads increases, and Apdex index value and level of user's satisfaction are significantly reduced under different load levels, mainly for the IBM provider. Besides, the error rate rises substantially at 400 threads, with more critical results for IBM and Amazon providers. Conclusions: Providers show distinct differences across metrics, and the data collected during testings reinforced the potential particularities of cloud infrastructure services for the choice of a provider. Although preliminary, such results can support software companies in selecting a proper infrastructure provider according to particular requirements.

查看原文本刊更多论文

不同负载水平下云基础设施供应商的实验性能评估

背景:性能测试可以估计请求下web服务的容量。对于部署特定云应用程序的理想云服务基础设施的决策是具有挑战性的。目标:调查不同负载水平下云基础设施提供商的性能。方法:一项实验研究评估了亚马逊、Azure、谷歌和IBM云基础设施提供商在基础设施即服务(IaaS)视角下的性能。结果:结果表明，当同时处理多达300个线程时，大多数提供程序在响应时间和延迟方面的性能令人满意。但是，这种影响会随着线程数量的增加而减小，而且Apdex索引值和用户满意度在不同的负载水平下会显著降低，这主要针对IBM提供商。此外，错误率在400个线程时大幅上升，IBM和Amazon提供商的结果更为关键。结论:供应商在各项指标中表现出明显的差异，在测试期间收集的数据强化了云基础设施服务在选择供应商时的潜在特殊性。虽然是初步的，但是这样的结果可以支持软件公司根据特定的需求选择合适的基础设施提供商。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2021 XLVII Latin American Computing Conference (CLEI)

自引率

0.00%

发文量