G. Papadimitriou, M. Kiran, Cong Wang, A. Mandal, E. Deelman
{"title":"Training Classifiers to Identify TCP Signatures in Scientific Workflows","authors":"G. Papadimitriou, M. Kiran, Cong Wang, A. Mandal, E. Deelman","doi":"10.1109/INDIS49552.2019.00012","DOIUrl":"https://doi.org/10.1109/INDIS49552.2019.00012","url":null,"abstract":"Identifying network anomalies is an important mea- sure to ensure reliability and quality of data transfers among facilities. Scientific workflows in particular heavily rely on good network performance to ensure their smooth executions. In this paper, we present a lightweight classifier system that is able to recognize anomalous TCP transfers. Using random forest trees and labeled data sets, we evaluate the classifier with real workflow transfers for ground truth data. Our studies reveal that various TCP congestion algorithms behave differently in anomalous conditions. We show that training classifiers on these separately can aid detection in network performance deterioration. Results reveal that our classifiers are able to better predict anomalous flows for TCP Reno and Hamilton compared to Cubic and BBR, due to the manner how their congestion control algorithms handle the anomalies.","PeriodicalId":145461,"journal":{"name":"2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134160500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Hop Recording and Forwarding State Logging: Two Implementations for Path Tracking in P4","authors":"Silke Knossen, Joseph Hill, P. Grosso","doi":"10.1109/INDIS49552.2019.00010","DOIUrl":"https://doi.org/10.1109/INDIS49552.2019.00010","url":null,"abstract":"Full information on the path travelled by packets is extremely important for network management and network security. We implemented two path tracking methods in hardware with P4. The first approach tracks a packet's path by recording each node along the path of a packet (hop recording). The complete path a packet took can be extracted from the packet in the last node of the path. The second approach tracks a packet's path by logging the forwarding state of a network (forwarding state logging). The complete path can be reconstructed based on the node where the packet entered a network. We conducted experiments with the two implemented approaches and showed that the paths of the packets are reconstructed correctly. The advantage of using P4 is that the control plane only gets involved when the path of a packet is reconstructed. We finally show how our work provides a working tool in P4 networks that can be used to gain deep insights in traffic patterns.","PeriodicalId":145461,"journal":{"name":"2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127447480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Co-Scheduling of Advance and Immediate Bandwidth Reservations for Inter-Data Center Transfer","authors":"Aiqin Hou, C. Wu, Liudong Zuo, Dawei Quan, Yangyang Li, Mengxia Zhu, Q. Duan, Dingyi Fang","doi":"10.1109/INDIS49552.2019.00009","DOIUrl":"https://doi.org/10.1109/INDIS49552.2019.00009","url":null,"abstract":"As scientific applications and business services increasingly migrate to clouds, big data of various types with different priorities need to be transferred between geographically distributed cloud-based data centers. It has become a critical task for Cloud Service Providers (CSP) to fully utilize the expensive bandwidth resources of the links connecting such data centers while guaranteeing users’ Quality of Experience (QoE). Most high-performance networks based on software-defined networking (SDN) provide the capability of advance bandwidth reservation. This paper focuses on the scheduling of multiple user requests of two different types with different priorities, namely, advance bandwidth reservation with a lower priority and immediate bandwidth reservation with a higher priority, to maximize the total satisfaction of user requests.We formulate this co-scheduling problem as a generic optimization problem, which is shown to be NP-complete. We design a heuristic algorithm to maximize the number of successfully scheduled requests and minimize the number of preempted advance reservation requests, while minimizing the completion time of each request. Extensive simulation results show that our scheduling scheme significantly outperforms greedy approaches in terms of user satisfaction degree.","PeriodicalId":145461,"journal":{"name":"2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"204 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134130158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sample Transfer Optimization with Adaptive Deep Neural Network","authors":"Hemanta Sapkota, M. Arifuzzaman, Engin Arslan","doi":"10.1109/INDIS49552.2019.00013","DOIUrl":"https://doi.org/10.1109/INDIS49552.2019.00013","url":null,"abstract":"Application-layer transfer configurations play a crucial role in achieving desirable performance in high-speed networks. However, finding the optimal configuration for a given transfer task is a difficult problem as it depends on various factors including dataset characteristics, network settings, and background traffic. The state-of-the-art transfer tuning solutions rely on real-time sample transfers to evaluate various configurations and estimate the optimal one. However, existing approaches to run sample transfers incur high delay and measurement errors, thus significantly limit the efficiency of the transfer tuning algorithms. In this paper, we introduce adaptive feed forward deep neural network (DNN) to minimize the error rate of sample transfers without increasing their execution time. We ran 115K file transfers in four different high-speed networks and used their logs to train an adaptive DNN that can quickly and accurately predict the throughput of sample transfers by analyzing instantaneous throughput values. The results gathered in various networks with rich set of transfer configurations indicate that the proposed model reduces error rate by up to 50% compared to the state-of-the-art solutions while keeping the execution time low. We also show that one can further reduce delay or error rate by tuning hyperparameters of the model to meet specific needs of user or application. Finally, transfer learning analysis reveals that the model developed in one network would yield accurate results in other networks with similar transfer convergence characteristics, alleviating the needs to run an extensive data collection and model derivation efforts for each network.","PeriodicalId":145461,"journal":{"name":"2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127608750","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
N. Rao, N. Imam, Zhengchun Liu, R. Kettimuthu, Ian T Foster
{"title":"Estimation of RTT and Loss Rate of Wide-Area Connections Using MPI Measurements","authors":"N. Rao, N. Imam, Zhengchun Liu, R. Kettimuthu, Ian T Foster","doi":"10.1109/INDIS49552.2019.00008","DOIUrl":"https://doi.org/10.1109/INDIS49552.2019.00008","url":null,"abstract":"Abstract—Scientific computations are expected to be increasingly distributed across wide-area networks, and the Message Passing Interface (MPI) has been shown to scale to support their communications over long distances. These computations should account for certain network parameters to ensure an effective execution, for example, by avoiding highly congested and long connections. The execution times of MPI basic operations reflect the connection parameters, including the Round Trip Time (RTT) and loss rate. We describe five machine leaning methods to estimate the connection RTT and loss rate using execution times of MPI basic operations. We utilize execution time measurements of MPI Sendrecv operations collected over emulated 10 Gbps connections with 0-366 ms round-trip times, wherein the longest connection spans the globe, under up to 20% periodic losses. These methods provide disparate, namely, linear and non-linear, and smooth and non-smooth, estimates of RTT and loss rate. Our results show that accurate estimates can be generated at low loss rates but they become inaccurate at loss rates 10% and higher. Overall, these results constitute a case study of the strengths and limitations of machine learning methods in inferring network-level parameters using application-level measurements.","PeriodicalId":145461,"journal":{"name":"2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121604883","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Se-young Yu, J. Chen, F. Yeh, J. Mambretti, Xiao Wang, Anna Giannakou, Eric Pouyoul, M. Lyonnais
{"title":"SCinet DTN-as-a-Service Framework","authors":"Se-young Yu, J. Chen, F. Yeh, J. Mambretti, Xiao Wang, Anna Giannakou, Eric Pouyoul, M. Lyonnais","doi":"10.1109/INDIS49552.2019.00006","DOIUrl":"https://doi.org/10.1109/INDIS49552.2019.00006","url":null,"abstract":"Transferring big data over Wide Area Networks (WANs) is challenging because optimization is dependent on the specifics of multiple parameters. Network services, paths, and technologies have different characteristics, including loss rate, latency, and available capacity. Yet, frameworks currently used to configure and orchestrate transfer systems, measure performance, and analyze results have limited capabilities. We propose a framework, DTN-as-a-Service (DaaS), for high-performance network data transfers using and integration of techniques, including virtualization, network provisioning, and performance data analysis. This framework has a modular design for supporting multiple transfer tools, optimizers and orchestrators for the data transfer environment, including textit{Docker} and textit{Kubernetes}. We present a textit{Jupyter} based workflow for high-speed network data transfer in data-intensive science and evaluate the performance of the transfer with a simple programmable visualizer implemented in the framework. This framework has been implemented as a prototype at two recent SC supercomputing conferences. With the increase in the number and the capacity of WAN links at the conferences (multiple 100 Gbps WAN circuits), the challenges involved in setting up, testing, debugging, verifying and running applications on high-performance systems connecting to the conference SCinet WAN circuits also increase. The SCinet implementation of the DaaS framework for the conference community allowed users to control hardware, software, and network infrastructure for high-speed network data transfer, primarily for large scale applications. Through the evaluation of the framework in our test setup, we demonstrated that NVMe over Fabrics with TCP is twice as efficient compared to using conventional TCP in high-speed NVMe-to-NVMe transfers. We also implemented a 400 Gbps LAN experiment to evaluate the DaaS framework.","PeriodicalId":145461,"journal":{"name":"2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"91 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123480602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daniel Nägele, Christopher B. Hauser, Leonard Bradatsch, S. Wesner
{"title":"bwNetFlow: A Customizable Multi-Tenant Flow Processing Platform for Transit Providers","authors":"Daniel Nägele, Christopher B. Hauser, Leonard Bradatsch, S. Wesner","doi":"10.1109/INDIS49552.2019.00007","DOIUrl":"https://doi.org/10.1109/INDIS49552.2019.00007","url":null,"abstract":"In times of increasing bandwidth demands, network operators strive for increased visibility of their network's utilization as well as an indication of the legitimacy of traffic processed across network nodes. Additionally, the detection and mitigation of illegitimate or malicious traffic such as denial of service attacks remains a current and persistently active field of research. Flow-based network monitoring can provide this information live from any network interface. This paper introduces a flow processing platform meant to receive flow information from border interfaces and distribute the acquired information to specialized applications. Transit providers deploying our platform can use this information directly, but also provide all interested customers or network entities with the specific subset concerning them. Between collecting and redistributing the flow information, our platform offers different methods of enrichment using a variety of sources allowing for views superior to plain Netflow records. However, a provided tool can reencode and reexport standard Netflow to ensure compatibility and allow for seamless integration of customer-specific streams into preexisting setups. This platform's components allow the enrichment, division and anonymization of flow data to a number of highly customized streams for any type of application, either on a customer-specific or a network-wide provider level. Applications include the conversion of flow data for time-series databases and the accompanying dashboards, the detection of DDoS attacks or other high-traffic situation on any network level, the identification of faulty network routing policies, or any other use case conceivable on regular flow data, but within an arbitrary network scope.","PeriodicalId":145461,"journal":{"name":"2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134120807","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jordi Ros-Giralt, L. Tassiulas, Sruthi Yellamraju, Atul Bohara, R. Lethin, Josie Li, Ying Lin, Yuanlong Tan, M. Veeraraghavan, Yuang Jiang
{"title":"G2: A Network Optimization Framework for High-Precision Analysis of Bottleneck and Flow Performance","authors":"Jordi Ros-Giralt, L. Tassiulas, Sruthi Yellamraju, Atul Bohara, R. Lethin, Josie Li, Ying Lin, Yuanlong Tan, M. Veeraraghavan, Yuang Jiang","doi":"10.1109/INDIS49552.2019.00011","DOIUrl":"https://doi.org/10.1109/INDIS49552.2019.00011","url":null,"abstract":"Congestion control algorithms for data networks have been the subject of intense research for the last three decades. While most of the work has focused around the characterization of a flow's bottleneck link, understanding the interactions amongst links and the ripple effects that perturbations in a link can cause on the rest of the network has remained much less understood. The Theory of Bottleneck Ordering is a recently developed mathematical framework that reveals the bottleneck structure of a network and provides a model to understand such effects. In this paper we present G2, the first operational network optimization framework that utilizes this new theoretical framework to characterize with high-precision the performance of bottlenecks and flows. G2 generates an interactive graph structure that describes how perturbations in links and flows propagate, providing operators new optimization insights and traffic engineering recommendations to help improve network performance. We provide a description of the G2 implementation and a set of experiments using production TCP/IP code to demonstrate its operational efficacy.","PeriodicalId":145461,"journal":{"name":"2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128829974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}