Is Singularity-based Container Technology Ready for Running MPI Applications on HPC Clouds?

Jie Zhang, Xiaoyi Lu, D. Panda
{"title":"Is Singularity-based Container Technology Ready for Running MPI Applications on HPC Clouds?","authors":"Jie Zhang, Xiaoyi Lu, D. Panda","doi":"10.1145/3147213.3147231","DOIUrl":null,"url":null,"abstract":"The Message Passing Interface (MPI) standard has become the de facto programming model for parallel computing with the last 25-year continuous community effort. With the development of building efficient HPC clouds, more and more MPI-based HPC applications start running on cloud-based environments. Singularity is one of the most attractive container technologies to build HPC clouds due to the claimed reproducible environments across the HPC centers. However, our investigations in the literature show that there is a lack of a systematical study on evaluating the performance of Singularity with various benchmarks and applications on different types of HPC platforms. Without these studies, it remains difficult to tell the community whether Singularity-based container technology is ready or not for running MPI applications on HPC clouds to gain desired performance. To fill this gap in the literature, as a third-party, we first propose a four-dimension evaluation methodology to cover various aspects and based on that, we conduct extensive studies on evaluating the performance of Singularity on modern processors, and high-performance interconnects. Performance results prove that Singularity-based container technology can achieve near-native performance for both Intel Xeon and Intel Xeon Knights Landing (KNL) platforms with different memory access modes (i.e., cache, flat). Singularity also shows very little overhead for running MPI-based HPC applications on both Omni-Path and InfiniBand networks. With the verification of our results, we believe that Singularity can be used for building next-generation HPC clouds with near-native performance as well as desired cloud features such as easy management and deployment.","PeriodicalId":341011,"journal":{"name":"Proceedings of the10th International Conference on Utility and Cloud Computing","volume":"57 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2017-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"26","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the10th International Conference on Utility and Cloud Computing","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3147213.3147231","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 26

Abstract

The Message Passing Interface (MPI) standard has become the de facto programming model for parallel computing with the last 25-year continuous community effort. With the development of building efficient HPC clouds, more and more MPI-based HPC applications start running on cloud-based environments. Singularity is one of the most attractive container technologies to build HPC clouds due to the claimed reproducible environments across the HPC centers. However, our investigations in the literature show that there is a lack of a systematical study on evaluating the performance of Singularity with various benchmarks and applications on different types of HPC platforms. Without these studies, it remains difficult to tell the community whether Singularity-based container technology is ready or not for running MPI applications on HPC clouds to gain desired performance. To fill this gap in the literature, as a third-party, we first propose a four-dimension evaluation methodology to cover various aspects and based on that, we conduct extensive studies on evaluating the performance of Singularity on modern processors, and high-performance interconnects. Performance results prove that Singularity-based container technology can achieve near-native performance for both Intel Xeon and Intel Xeon Knights Landing (KNL) platforms with different memory access modes (i.e., cache, flat). Singularity also shows very little overhead for running MPI-based HPC applications on both Omni-Path and InfiniBand networks. With the verification of our results, we believe that Singularity can be used for building next-generation HPC clouds with near-native performance as well as desired cloud features such as easy management and deployment.
基于奇点的容器技术为在HPC云上运行MPI应用做好准备了吗?
经过过去25年社区的不懈努力,消息传递接口(Message Passing Interface, MPI)标准已经成为并行计算事实上的编程模型。随着构建高效HPC云的发展,越来越多基于mpi的HPC应用开始在云环境中运行。Singularity是构建HPC云的最具吸引力的容器技术之一,因为它可以跨HPC中心实现可复制的环境。然而,我们在文献中的调查表明,在不同类型的HPC平台上使用各种基准和应用程序来评估Singularity的性能缺乏系统的研究。如果没有这些研究,就很难告诉社区基于奇点的容器技术是否已经准备好在HPC云上运行MPI应用程序以获得期望的性能。为了填补这一文献空白,作为第三方,我们首先提出了一个涵盖各个方面的四维评估方法,并在此基础上对现代处理器和高性能互连的奇点性能评估进行了广泛的研究。性能结果证明,基于奇点的容器技术可以在英特尔至强和英特尔至强骑士登陆(KNL)平台上实现近乎原生的性能,并具有不同的内存访问模式(即缓存、平面)。Singularity还显示,在Omni-Path和InfiniBand网络上运行基于mpi的HPC应用程序的开销非常小。通过对结果的验证,我们相信Singularity可以用于构建下一代高性能计算云,具有接近原生性能的云以及期望的云特性,如易于管理和部署。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信