2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)最新文献

[Copyright notice] (版权)

2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS) Pub Date : 2020-11-01 DOI: 10.1109/indis51933.2020.00002

引用次数: 0

Computing Bottleneck Structures at Scale for High-Precision Network Performance Analysis 高精度网络性能分析的大规模计算瓶颈结构

2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS) Pub Date : 2020-11-01 DOI: 10.1109/INDIS51933.2020.00012

Noah Amsel, Jordi Ros-Giralt, Sruthi Yellamraju, J. Ezick, Brendan von Hofe, Alison Ryan, R. Lethin

{"title":"Computing Bottleneck Structures at Scale for High-Precision Network Performance Analysis","authors":"Noah Amsel, Jordi Ros-Giralt, Sruthi Yellamraju, J. Ezick, Brendan von Hofe, Alison Ryan, R. Lethin","doi":"10.1109/INDIS51933.2020.00012","DOIUrl":"https://doi.org/10.1109/INDIS51933.2020.00012","url":null,"abstract":"The Theory of Bottleneck Structures is a recently-developed framework for studying the performance of data networks. It describes how local perturbations in one part of the network propagate and interact with others. This framework is a powerful analytical tool that allows network operators to make accurate predictions about network behavior and thereby optimize performance. Previous work implemented a software package for bottleneck structure analysis, but applied it only to toy examples. In this work, we introduce the first software package capable of scaling bottleneck structure analysis to production-size networks. We benchmark our system using logs from ESnet, the Department of Energy's high-performance data network that connects research institutions in the U.S. Using the previously published tool as a baseline, we demonstrate that our system achieves vastly improved performance, constructing the bottleneck structure graphs in 0.21 s and calculating link derivatives in 0.09 s on average. We also study the asymptotic complexity of our core algorithms, demonstrating good scaling properties and strong agreement with theoretical bounds. These results indicate that our new software package can maintain its fast performance when applied to even larger networks. They also show that our software is efficient enough to analyze rapidly changing networks in real time. Overall, we demonstrate the feasibility of applying bottleneck structure analysis to solve practical problems in large, real-world data networks.","PeriodicalId":394752,"journal":{"name":"2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128750792","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Application Aware Software Defined Flows of Workflow Ensembles 应用感知软件定义工作流集成的流程

2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS) Pub Date : 2020-11-01 DOI: 10.1109/INDIS51933.2020.00007

G. Papadimitriou, Eric J. Lyons, Cong Wang, Komal Thareja, Ryan Tanaka, P. Ruth, J. J. Villalobos, I. Rodero, E. Deelman, M. Zink, A. Mandal

{"title":"Application Aware Software Defined Flows of Workflow Ensembles","authors":"G. Papadimitriou, Eric J. Lyons, Cong Wang, Komal Thareja, Ryan Tanaka, P. Ruth, J. J. Villalobos, I. Rodero, E. Deelman, M. Zink, A. Mandal","doi":"10.1109/INDIS51933.2020.00007","DOIUrl":"https://doi.org/10.1109/INDIS51933.2020.00007","url":null,"abstract":"Computational science depends on complex, data intensive applications operating on datasets from a variety of scientific instruments. A major challenge is the integration of data into the scientist's workflow. Recent advances in dynamic, networked cloud resources provide the building blocks to construct reconfiguration, end-to-end infrastructure that can increase scientific productivity, but applications are not taking advantage of them. In our previous work, we introduced Dy-N amo, that enabled CASA scientists to improve the efficiency of their operations and effortlessly leverage capabilities of the cloud resources available to them that previously remained underutilized. However, the provided workflow automation did not satisfy all the operational requirements of CASA. Custom scripts were still in production to manage workflow triggering, while multiple layer2 connections would have to be allocated to maintain network QoS requirements. In this work, we enhance the DyNamo system with ensemble workflow management capabilities, end-to-end infrastructure monitoring, as well as more advanced network manipulation mechanisms. To accommodate CASA's operational needs we also extended the newly integrated Pegasus Ensemble Manager with file and time based triggering functionality, that improves managing workflow ensembles. Additionally, Virtual Software Defined Exchange (vSDX) capabilities have been extended, enabling link adaptation, flow prioritization and traffic control between endpoints. We evaluate the effects of the DyNamo's vSDX policies by using two CASA workflow ensembles competing for network resources, and we show that traffic shaping of the ensembles can lead to a fairer use of the network links.","PeriodicalId":394752,"journal":{"name":"2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134647837","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

An Evaluation of Ethernet Performance for Scientific Workloads 科学工作负载下的以太网性能评估

2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS) Pub Date : 2020-11-01 DOI: 10.1109/INDIS51933.2020.00011

J. Kenny, Jeremiah J. Wilke, C. Ulmer, Gavin Baker, Samuel Knight, J. A. Friesen

{"title":"An Evaluation of Ethernet Performance for Scientific Workloads","authors":"J. Kenny, Jeremiah J. Wilke, C. Ulmer, Gavin Baker, Samuel Knight, J. A. Friesen","doi":"10.1109/INDIS51933.2020.00011","DOIUrl":"https://doi.org/10.1109/INDIS51933.2020.00011","url":null,"abstract":"Priority-based Flow Control (PFC), RDMA over Converged Ethernet (RoCE) and Enhanced Transmission Selection (ETS) are three enhancements to Ethernet networks which allow increased performance and may make Ethernet attractive for systems supporting a diverse scientific workload. We constructed a 96-node testbed cluster with a 100 Gb/s Ethernet network configured as a tapered fat tree. Tests representing important network operating conditions were completed and we provide an analysis of these performance results. RoCE running over a PFC-enabled network was found to significantly increase performance for both bandwidth-sensitive and latency-sensitive applications when compared to TCP. Additionally, a case study of interfering applications showed that ETS can prevent starvation of network traffic for latency-sensitive applications running on congested networks. We did not encounter any notable performance limitations for our Ethernet testbed, but we found that practical disadvantages still tip the balance towards traditional HPC networks unless a system design is driven by additional external requirements.","PeriodicalId":394752,"journal":{"name":"2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123575756","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Using P4 and RDMA to collect telemetry data 使用P4和RDMA采集遥测数据

2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS) Pub Date : 2020-11-01 DOI: 10.1109/INDIS51933.2020.00006

Rutger Beltman, Silke Knossen, Joseph Hill, P. Grosso

引用次数: 5

The NetSage Measurement Framework: Design, Development, and Discoveries NetSage测量框架:设计、开发和发现

2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS) Pub Date : 2020-11-01 DOI: 10.1109/INDIS51933.2020.00010

Katrina Turner, Mahesh Khanal, Tyson Seto-Mook, Alberto Gonzalez, J. Leigh, A. Lake, Sartaj Singh Baveha, Samir Faci, B. Tierney, D. Doyle, L. Ensman, J. Schopf, Doug Southworth, E. Balas

引用次数: 3

ROBIN (RuciO/BIgData Express/SENSE) A Next-Generation High-Performance Data Service Platform ROBIN (RuciO/BIgData Express/SENSE)下一代高性能数据服务平台

2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS) Pub Date : 2020-11-01 DOI: 10.1109/INDIS51933.2020.00009

Wenji Wu, Liang Zhang, Qiming Lu, P. DeMar, R. Illingworth, J. Mambretti, Se-young Yu, J. Chen, I. Monga, Xi Yang, T. Lehman, C. Guok, J. Macauley

{"title":"ROBIN (RuciO/BIgData Express/SENSE) A Next-Generation High-Performance Data Service Platform","authors":"Wenji Wu, Liang Zhang, Qiming Lu, P. DeMar, R. Illingworth, J. Mambretti, Se-young Yu, J. Chen, I. Monga, Xi Yang, T. Lehman, C. Guok, J. Macauley","doi":"10.1109/INDIS51933.2020.00009","DOIUrl":"https://doi.org/10.1109/INDIS51933.2020.00009","url":null,"abstract":"Increasingly, scientific research discovery depends on the analysis of extremely large amounts of data. This data is gathered through very large scientific instruments, very large numbers of small devices, and mid-scale instruments. These devices generate exponentially increasing volumes of data. Such data is often obtained, analyzed, stored, visualized, and transferred at and among sites world-wide by large collaborations of science communities. Managing and moving extremely large volumes of data across today's networks is a special multidimensional challenge. Also, existing data management and movement services and tools often are inadequate to address all specific requirements. This paper describes the various components of this challenge and how those issues can be addressed by ROBIN, a unique comprehensive set of integrated services designed specifically for managing and moving extremely large amounts of data over long distances, e.g., thousands of miles around the globe. This paper also describes the results of initial experiments using that set of integrated services.","PeriodicalId":394752,"journal":{"name":"2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121959597","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Trial Deployment of a Reliable Network-Multicast Application across Internet2 跨Internet2可靠网络组播应用的试验部署

2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS) Pub Date : 2020-11-01 DOI: 10.1109/INDIS51933.2020.00008

Yuanlong Tan, M. Veeraraghavan, Hwajung Lee, S. Emmerson, J. Davidson

{"title":"A Trial Deployment of a Reliable Network-Multicast Application across Internet2","authors":"Yuanlong Tan, M. Veeraraghavan, Hwajung Lee, S. Emmerson, J. Davidson","doi":"10.1109/INDIS51933.2020.00008","DOIUrl":"https://doi.org/10.1109/INDIS51933.2020.00008","url":null,"abstract":"A continuing trend in many scientific disciplines is the growth in the volume of data collected by scientific instruments and the desire to rapidly and efficiently distribute this data to the scientific community. Transferring these large data sets to a geographically distributed research community consumes significant network bandwidth. As both the data volume and number of subscribers grows, reliable network multicast is a promising approach to reduce the rate of growth of the bandwidth needed to support efficient data distribution. In prior work, we identified a need for reliable network multicast: scientists engaged in atmospheric research subscribing to meteorological file-streams. Specifically, the University Cooperation Atmospheric Research (UCAR) uses the Local Data Manager (LDM) to disseminate data. This work describes a trial deployment of a multicast-enabled LDM, in which eight university campuses are connected via corresponding regional Research-and-Education Networks (RENs) and Internet2. Using this deployment, we evaluated the new version of LDM, LDM7, which uses network multicast with a reliable transport protocol, and leverages Layer-2 (L2) multipoint Virtual LAN (VLANIMPLS). A performance monitoring system was deployed to collect real-time performance of LDM7, which showed that our proof-of-concept prototype worked significantly better than the current production LDM, LDM6, in two ways: (i) LDM7 can distribute file streams faster than LDM6. With six subscribers, an almost 22-fold improvement was observed with LDM7 at 100 Mbps. And (ii) to achieve a similar performance, LDM7 significantly reduces the need for bandwidth, which reduced the bandwidth requirement by about 90% over LDM6 to achieve 20 Mbps average throughput across four subscribers.","PeriodicalId":394752,"journal":{"name":"2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124066982","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0