J. Syst. Res.最新文献

{"title":"[SoK] The Great GAN Bake Off, An Extensive Systematic Evaluation of Generative Adversarial Network Architectures for Time Series Synthesis","authors":"Mark Leznik, Arne Lochner, S. Wesner, Jörg Domaschka","doi":"10.5070/sr32159045","DOIUrl":"https://doi.org/10.5070/sr32159045","url":null,"abstract":"There is no standard approach to compare the success of different neural network architectures utilized for time series synthesis. This hinders the evaluation and decision, which architecture should be leveraged for an unknown data set. We propose a combination of metrics, which empirically evaluate the performance of neural network architectures trained for time series synthesis. With these measurements we are able to account for temporal correlations, spatial correlations and mode collapse issues within the generated time series. We further investigate the interaction of different generator and discriminator architectures between each other. The considered architectures include recurrent neural networks, temporal convolutional networks and transformer-based networks. So far, the application of transformer-based models is limited for time series synthesis. Hence, we propose a new transformer-based architecture, which is able to synthesise time series. We evaluate the proposed architectures and their comobinations in over 500 experiments, amounting to over 2500 computing hours. We provide results for four data data sets, one univariate and three multivariate. The data sets vary with regard to length, patterns in temporal and spatial correlations. We use our metrics to compare the performance of generative adversarial network architectures for time series synthesis. To verify our findings we utilize quantitative and qualitative evaluations. Our results indicate that temporal convolutional networks outperform recurrent neural network and transformer based approaches with regard to fidelity and flexibility of the generated data. Temporal convolutional network architecture are the most stable architecture for a mode collapse prone data set. The performance of the transformer models strongly depends on the data set characteristics, it struggled to synthesise data sets with high temporal and spatial correlations. Discriminators with recurrent network architectures suffered immensely from vanishing gradients. We also show, that the performance of the generative adversarial networks depends more on the discriminator part rather than the generator part.","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114622329","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"[Solution] End-to-end Scheduling of Real-time Task Pipelines on Multiprocessors","authors":"Soham Sinha, R. West","doi":"10.5070/sr32158647","DOIUrl":"https://doi.org/10.5070/sr32158647","url":null,"abstract":"","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132639705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"[Solution] Prepare your video for streaming with Segue","authors":"Melissa Licciardello, Lukas Humbel, Fabian Rohr, Maximilian Grüner, Ankit Singla","doi":"10.5070/sr32158113","DOIUrl":"https://doi.org/10.5070/sr32158113","url":null,"abstract":"","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115743872","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"[SoK] Identifying Mismatches Between Microservice Testbeds and Industrial Perceptions of Microservices","authors":"Vishwanath Seshagiri, Darby Huye, Lan Liu, Avani Wildani, Raja R. Sambasivan","doi":"10.5070/sr32157839","DOIUrl":"https://doi.org/10.5070/sr32157839","url":null,"abstract":"Industrial microservice architectures vary so wildly in their characteristics, such as size or communication method, that comparing systems is difficult and often leads to confusion and misinterpretation. In contrast, the academic testbeds used to conduct microservices research employ a very constrained set of design choices. This lack of systemization in these key design choices when developing microservice architectures has led to uncertainty over how to use experiments from testbeds to inform practical deployments and indeed whether this should be done at all. We conduct semi-structured interviews with industry participants to understand the representativeness of existing testbeds’ design choices. Surprising results included the presence of cycles in industry deployments, as well as a lack of clarity about the presence of hierarchies. We then systematize the possible design choices we learned about from the interviews, and identify important mismatches between our interview results and testbeds’ designs that will inform future, more representative testbeds.","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"82 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126040316","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"[Solution] Mir-BFT: Scalable and Robust BFT for Decentralized Networks","authors":"C. Stathakopoulou, Tudor David, M. Pavlovic, M. Vukolic","doi":"10.5070/sr32159278","DOIUrl":"https://doi.org/10.5070/sr32159278","url":null,"abstract":"The paper presents Mir-BTF a Byzantine fault-tolerant total order broadcast system that uses multiple leaders to increase the total throughput. The key contribution of Mir-BTF is the novel hash-based bucketing of requests coupled with leader rotation such that neither request are duplicated nor requests are consistently censored. This allows Mir-BTF to achieves up to 60.000 transactions per second throughput in a deployment of 100 replicas. The reviewers appreciated the simplicity of building Mir-BTF atop PBFT making the protocol easier to be adopted. Mir-BTF is currently under adoption from a major cryptocurrency.","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128670835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"[Solution] ALCC: Migrating Congestion Control To The Application Layer In Cellular Networks","authors":"Y. Zaki, Rohail Asim, Muhammad Khan, Shiva R. Iyer, Talal Ahmad, Thomas Pötsch, Lakshmi Subramanian","doi":"10.5070/sr32156656","DOIUrl":"https://doi.org/10.5070/sr32156656","url":null,"abstract":"","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133932995","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"[Tool] Designing Replicable Networking Experiments With Triscale","authors":"Romain Jacob, Marco Zimmerling, C. Boano, L. Vanbever, L. Thiele","doi":"10.5070/sr31155408","DOIUrl":"https://doi.org/10.5070/sr31155408","url":null,"abstract":"When designing their performance evaluations, networking researchers often encounter questions such as: How long should a run be? How many runs to perform? How to account for the variability across multiple runs? What statistical methods should be used to analyze the data? Despite their best intentions, researchers often answer these questions dif-ferently, thus impairing the replicability of their evaluations and the confidence in their results. In this paper, we propose a concrete methodology for the design and analysis of performance evaluations. Our approach hierarchically partitions the performance evaluation into three timescales, following the principle of separation of concerns. The idea is to understand, for each timescale, the temporal characteristics of variability sources, and then to apply rigorous statistical methods to derive performance results with quantifiable confidence in spite of the inherent variability. We implement this methodology in a software framework called TriScale . For each performance metric, TriScale computes a variability score that estimates, with a given confidence, how similar the results would be if the evaluation were repli-cated; in other words, TriScale quantifies the replicability of evaluations. We showcase the practicality and usefulness of TriScale on four different case studies demonstrating that TriScale helps to generalize and strengthen published results. Improving the standards of replicability in networking is a complex challenge. This paper is an important contribution to this endeavor; it provides networking researchers with a rational and concrete experimental methodology rooted in sound statistical foundations. The first of its kind.","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133538430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SoK: A Generalized Multi-Leader State Machine Replication Tutorial","authors":"Michael J. Whittaker, N. Giridharan, Adriana Szekeres, J. Hellerstein, I. Stoica","doi":"10.5070/sr31154817","DOIUrl":"https://doi.org/10.5070/sr31154817","url":null,"abstract":"MultiPaxos and Raft are the two most popular and widely deployed state machine replication protocols. There is a more sophisticated family of generalized multi-leader state machine replication protocols like EPaxos, Caesar, and Atlas that have better performance, but they are extremely complicated and hard to understand. Due to their complexity, they have seen little to no industry adoption, and academically there has been a lack of clarity in analyzing, comparing, and extending the protocols. This paper is a tutorial on generalized multi-leader protocols. We explain why the protocols work the way they do, what they have in common, where they differ, which parts of the protocols are straightforward, which are more subtle than they appear, and so on. In doing so, we present four new generalized multi-leader protocols, identify key insights into existing protocols, and taxonomize the space.","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125930213","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SoK: Function-As-A-Service: From An Application Developer's Perspective","authors":"A. Raza, I. Matta, Nabeel Akhtar, Vasiliki Kalavri, Vatche Isahagian","doi":"10.5070/sr31154815","DOIUrl":"https://doi.org/10.5070/sr31154815","url":null,"abstract":"In the past few years, FaaS has gained significant popularity and became a go-to choice for deploying cloud applications and micro-services. FaaS with its unique ‘pay as you go’ pricing model and key performance benefits over other cloud services, offers an easy and intuitive programming model to build cloud applications. In this model, a developer focuses on writing the code of the application while infrastructure management is left to the cloud provider who is responsible for the underlying resources, security, isolation, and scaling of the application. Recently, a number of commercial and open-source FaaS platforms have emerged, offering a wide range of features to application developers. In this paper, first, we present measurement studies demystifying various features and performance of commercial and open-source FaaS platforms that can help developers with deploying and configuring their serverless applications. Second, we discuss the distinct performance and cost benefits of FaaS and interesting use cases that leverage the performance, cost, or both aspects of FaaS. Lastly, we discuss challenges a developer may face while developing or deploying a serverless application. We also discuss state of the art solutions and open problems.","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"394 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115223914","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"[Solution] Matchmaker Paxos: A Reconfigurable Consensus Protocol","authors":"Michael J. Whittaker, N. Giridharan, Adriana Szekeres, J. Hellerstein, H. Howard, Faisal Nawab, I. Stoica","doi":"10.5070/sr31154842","DOIUrl":"https://doi.org/10.5070/sr31154842","url":null,"abstract":"The paper presents Matchmaker Paxos/Multi-Paxos, a crash fault-tolerant consensus implementation and state machine replication system with vertical reconfiguration. The main contribution is the reconfiguration protocol, a critical component of Consensus implementations that is often overlooked by the research community, but that is critical in practice. Reviewers share the same common feedback, the paper is relevant and represents a valuable addition to the literature of high-performance reconfiguration techniques for consensus. Abstract State machine replication protocols, like MultiPaxos and Raft, are at the heart of numerous distributed systems. To tolerate machine failures, these protocols must replace failed machines with new machines, a process known as reconfiguration. Reconfiguration has become increasingly important over time as the need for frequent reconfiguration has grown. Despite this, reconfiguration has largely been neglected in the literature. In this paper, we present Matchmaker Paxos and Matchmaker MultiPaxos, a reconfigurable consensus and state machine replication protocol respectively. Our protocols can perform a reconfiguration with little to no impact on the latency or throughput of command processing; they can perform a reconfiguration in a few milliseconds; and they present a framework that can be generalized to other replication protocols in a way that previous reconfiguration techniques can not. We provide proofs of correctness for the protocols and optimizations, and present empirical results from an open source implementation showing that throughput and latency do not change significantly during a reconfiguration.","PeriodicalId":363427,"journal":{"name":"J. Syst. Res.","volume":"41 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129611419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}