{"title":"Automated quality-of-service-aware configuration of publish-subscribe systems at design-time","authors":"Thomas Fischer, A. Wahl, R. Lenz","doi":"10.1145/2611286.2611293","DOIUrl":"https://doi.org/10.1145/2611286.2611293","url":null,"abstract":"Configurable publish-subscribe middleware provides efficient support for the diverse Quality-of-Service (QoS) requirements of large-scale distributed applications. However, choosing the optimal middleware configuration to suit a specific application primarily remains a manual task within the responsibility of application developers. Existing configurable middleware approaches either do not allow extensive configuration or lack the possibility to specify QoS requirements in a domain-specific terminology. In this paper we present a novel methodology for the QoS-aware configuration of publish-subscribe middleware, which relies on discrete-event simulations and supervised learning techniques to automatically translate declarative descriptions of the application domain to an optimal configuration. The configuration is conducted at design-time to minimize possible run-time impacts. We implemented a design-time configurable middleware and evaluated the quality of the automated configuration workflow as well as the performance of event dissemination. Our results show that simulation experiments in conjunction with regression methods can be used to reliably derive optimal middleware configurations without further user interaction. We clarify that extensive design-time configurability does not compromise run-time performance and suits the requirements of demanding applications.","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"3 1","pages":"118-129"},"PeriodicalIF":0.0,"publicationDate":"2014-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76928481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anilkumar Vishnoi, Rishabh Poddar, V. Mann, Suparna Bhattacharya
{"title":"Effective switch memory management in OpenFlow networks","authors":"Anilkumar Vishnoi, Rishabh Poddar, V. Mann, Suparna Bhattacharya","doi":"10.1145/2611286.2611301","DOIUrl":"https://doi.org/10.1145/2611286.2611301","url":null,"abstract":"OpenFlow networks require installation of flow rules in a limited capacity switch memory (Ternary Content Addressable Memory or TCAMs, in particular) from a logically centralized controller. A controller can manage the switch memory in an OpenFlow network through events that are generated by the switch at discrete time intervals. Recent studies have shown that data centers can have up to 10,000 network flows per second per server rack today. Increasing the TCAM size to accommodate these large number of flow rules is not a viable solution since TCAM is costly and power hungry. Current OpenFlow controllers handle this issue by installing flow rules with a default idle timeout after which the switch automatically evicts the rule from its TCAM. This results in inefficient usage of switch memory for short lived flows when the timeout is too high and in increased controller workload for frequent flows when the timeout is too low.\u0000 In this context, we present SmartTime - an OpenFlow controller system that combines an adaptive timeout heuristic to compute efficient idle timeouts with proactive eviction of flow rules, which results in effective utilization of TCAM space while ensuring that TCAM misses (or controller load) does not increase. To the best of our knowledge, SmartTime is the first real implementation of an intelligent flow management strategy in an OpenFlow controller that can be deployed in current OpenFlow networks. In our experiments using multiple real data center packet traces and cache sizes, SmartTime adaptive policy consistently outperformed the best performing static idle timeout policy or random eviction policy by up to 58% in terms of total cost.","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"44 1","pages":"177-188"},"PeriodicalIF":0.0,"publicationDate":"2014-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78723232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thomas S. Heinze, Zbigniew Jerzak, Gregor Hackenbroich, C. Fetzer
{"title":"Latency-aware elastic scaling for distributed data stream processing systems","authors":"Thomas S. Heinze, Zbigniew Jerzak, Gregor Hackenbroich, C. Fetzer","doi":"10.1145/2611286.2611294","DOIUrl":"https://doi.org/10.1145/2611286.2611294","url":null,"abstract":"Elastic scaling allows a data stream processing system to react to a dynamically changing query or event workload by automatically scaling in or out. Thereby, both unpredictable load peaks as well as underload situations can be handled. However, each scaling decision comes with a latency penalty due to the required operator movements. Therefore, in practice an elastic system might be able to improve the system utilization, however it is not able to provide latency guarantees defined by a service level agreement (SLA).\u0000 In this paper we introduce an elastic scaling system, which optimizes the utilization under certain latency constraints defined by a SLA. Specifically, we present a model, which estimates the latency spike created by a set of operator movements. We use this model to built a latency-aware elastic operator placement algorithm, which minimizes the number of latency violations. We show that our solution is able to reduce the 90th percentile of the end to end latency by up to 30% and reduce the number of latency violations by 50%. The achieved system utilization for our approach is comparable to a scaling strategy, which does not use latency as optimization target.","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"31 1","pages":"13-22"},"PeriodicalIF":0.0,"publicationDate":"2014-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81288887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The process-oriented event model (PoEM): a conceptual model for industrial events","authors":"O. Patri, V. Sorathia, A. Panangadan, V. Prasanna","doi":"10.1145/2611286.2611291","DOIUrl":"https://doi.org/10.1145/2611286.2611291","url":null,"abstract":"The paper presents a comprehensive theoretical framework for modeling events and semantics of event processing at a level of abstraction that captures the different processes in industrial applications but is not limited to a specific application domain. The model, called Process-oriented Event Model (PoEM), provides a formal approach to model real-world entities and their interrelationships, and specifies the process of moving from data streams to event detection to event-based goal planning. The model links event detection to states, actions, and roles, enabling event notification, filtering, context awareness and escalation. PoEM defines event and non-event concepts and combines information from them to build an event processing workflow. Usage of the PoEM model is illustrated in case studies from the oil and gas industry and maritime piracy events.","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"53 1","pages":"154-165"},"PeriodicalIF":0.0,"publicationDate":"2014-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82945219","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Verifying QoS properties of trusted data sensing","authors":"K. Ravindran, Arun Adiththan, A. Prabhakar","doi":"10.1145/2611286.2611317","DOIUrl":"https://doi.org/10.1145/2611286.2611317","url":null,"abstract":"Voting among replicated data collection devices achieves trusted data delivery to the end-user in a hostile environment. Failures may occur during data collection process: such as data corruptions by malicious devices and security/bandwidth attacks on data paths. How often a correct data is delivered to the user in a timely manner depicts the voting QoS. We study a model-based approach to inject faults in a data collection system for verification purposes. We advocate a stress-test of the system, i.e., subjecting the system to harsh conditions, to assess its resilience. The paper provides experimental studies of a voting system under injected faults, with message & time complexity analysis.","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"57 6","pages":"346-349"},"PeriodicalIF":0.0,"publicationDate":"2014-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91471522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Tutorial on the internet of everything","authors":"O. Etzion, Fabiana Fournier, Sarit Arcushin","doi":"10.1145/2611286.2611308","DOIUrl":"https://doi.org/10.1145/2611286.2611308","url":null,"abstract":"Analysts identified \"Internet of Everything\" (IoE), which is a generalization of the term Internet of Things as \"one of the major technology trends that will have a tremendous impact on life\". The term IoE stands for a collection of uniquely identified objects connected in an Internet-like structure, serving as sensors or actuators. However, Forrester Research stated recently that \"there is no Internet of Things, yet\". Indeed, what makes the traditional Internet pervasive is the ability to consume its content easily by eventually everybody. This is still not the case in the IoE. In this talk we discuss the main differences in thinking between the traditional Internet (which follows the request-response paradigm) and the Internet of Everything (which follows the sense & respond paradigm). This tutorial is intended to provide education in three major issues:\u0000 • What stands behind the IoE buzzword?\u0000 • What are the current and future applications of the IoE in different areas?\u0000 • What is missing today and what should researchers and practitioners concentrate on, to make the IoE really happen?","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"40 1","pages":"236-237"},"PeriodicalIF":0.0,"publicationDate":"2014-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81766158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Tudoran, O. Nano, I. Santos, Alexandru Costan, Hakan Soncu, L. Bougé, Gabriel Antoniu
{"title":"Achieving high throughput for large scale event streaming across geographically distributed data-centers with JetStream","authors":"R. Tudoran, O. Nano, I. Santos, Alexandru Costan, Hakan Soncu, L. Bougé, Gabriel Antoniu","doi":"10.1145/2611286.2611320","DOIUrl":"https://doi.org/10.1145/2611286.2611320","url":null,"abstract":"The increasing scale at which data processing is being performed nowadays calls for data management systems that enable high-performance data exchanges among geographically remote instances of large web services. In this demonstration we show how JetStream can increase the transfer rate of events which are streamed between geographically remote cloud data centers. The demonstration setup focuses on presenting how the binding can be done between JetStream and the event source on one hand and with the StreamInsight processing engine on the other hand. By considering a data source with an event generation rate that is variable in time, we demonstrate the importance of adapting the transfer scheme to the streaming context.","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"13 1","pages":"354-356"},"PeriodicalIF":0.0,"publicationDate":"2014-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85146655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A cloud-enabled coordination service for internet-scale OMG DDS applications","authors":"Kyoungho An, A. Gokhale","doi":"10.1145/2611286.2611323","DOIUrl":"https://doi.org/10.1145/2611286.2611323","url":null,"abstract":"The OMG Data Distribution Service (DDS), which is a standard specification for data-centric publish/subscribe communications, has shown promise for use in internet of things (IoT) applications because of its loosely coupled and scalable nature, and support for multiple QoS properties, such as reliable and real-time message delivery in dynamic environments. However, the current OMG DDS specification does not define coordination and discovery services for DDS message brokers, which are used in wide area network deployments of DDS. This paper describes preliminary research on a cloud-enabled coordination service for DDS message brokers, PubSubCoord, to overcome these limitations. Our approach provides a novel solution that brings together (a) ZooKeeper, which is used for the distributed coordination logic between message brokers, (b) DDS Routing Service, which is used to bridge DDS endpoints connected to different networks, and (c) BlueDove, which is used to provide a single-hop message delivery between brokers. Our design can support publishers and subscribers that dynamically join and leave their subnetworks.","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"34 1","pages":"310-313"},"PeriodicalIF":0.0,"publicationDate":"2014-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85241314","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thomas S. Heinze, Leonardo Aniello, Leonardo Querzoni, Zbigniew Jerzak
{"title":"Cloud-based data stream processing","authors":"Thomas S. Heinze, Leonardo Aniello, Leonardo Querzoni, Zbigniew Jerzak","doi":"10.1145/2611286.2611309","DOIUrl":"https://doi.org/10.1145/2611286.2611309","url":null,"abstract":"In this tutorial we present the results of recent research about the cloud enablement of data streaming systems. We illustrate, based on both industrial as well as academic prototypes, new emerging uses cases and research trends. Specifically, we focus on novel approaches for (1) scalability and (2) fault tolerance in large scale distributed streaming systems. In general, new fault tolerance mechanisms strive to be more robust and at the same time introduce less overhead. Novel load balancing approaches focus on elastic scaling over hundreds of instances based on the data and query workload. Finally, we present open challenges for the next generation of cloud-based data stream processing engines.","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"196 1","pages":"238-245"},"PeriodicalIF":0.0,"publicationDate":"2014-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79871232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Partition and compose: parallel complex event processing","authors":"Martin Hirzel","doi":"10.1145/2335484.2335506","DOIUrl":"https://doi.org/10.1145/2335484.2335506","url":null,"abstract":"Complex event processing uses patterns to detect composite events in streams of simple events. Typically, the events are logically partitioned by some key. For instance, the key can be the stock symbol in stock quotes, the author in tweets, the vehicle in transportation, or the patient in health-care. Composite event patterns often become meaningful only after partitioning. For instance, a pattern over stock quotes is typically meaningful over quotes for the same stock symbol. This paper proposes a pattern syntax and translation scheme organized around the notion of partitions. Besides making patterns meaningful, partitioning also benefits performance, since different keys can be processed in parallel. We have implemented partitioned parallel complex event processing as an extension to IBM's System S high-performance streaming platform. Our experiments with several benchmarks from finance and social media demonstrate processing speeds of up to 830,000 events per second, and substantial speedups for expensive patterns parallelized on multi-core machines as well as multi-machine clusters. Partitioning the event stream before detecting composite events makes event processing both more intuitive and parallel.","PeriodicalId":92123,"journal":{"name":"Proceedings of the ... International Workshop on Distributed Event-Based Systems. International Workshop on Distributed Event-Based Systems","volume":"22 1","pages":"191-200"},"PeriodicalIF":0.0,"publicationDate":"2012-07-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73957443","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}