{"title":"Load-aware shedding in stream processing systems","authors":"Nicolo Rivetti, Yann Busnel, Leonardo Querzoni","doi":"10.1145/2933267.2933311","DOIUrl":"https://doi.org/10.1145/2933267.2933311","url":null,"abstract":"Load shedding is a technique employed by stream processing systems to handle unpredictable spikes in the input load whenever available computing resources are not adequately provisioned. A load shedder drops tuples to keep the input load below a critical threshold and thus avoid tuple queuing and system trashing. In this paper we propose Load-Aware Shedding (LAS), a novel load shedding solution that drops tuples with the aim of maintaining queuing times below a tunable threshold. Tuple execution durations are estimated at runtime using efficient sketch data structures. We provide a theoretical analysis proving that LAS is an (ε, δ)-approximation of the optimal online load shedder and show its performance through a practical evaluation based both on simulations and on a running prototype.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128936917","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Kammoun, Syed Gillani, C. Gravier, Julien Subercaze
{"title":"High performance top-k processing of non-linear windows over data streams","authors":"A. Kammoun, Syed Gillani, C. Gravier, Julien Subercaze","doi":"10.1145/2933267.2933507","DOIUrl":"https://doi.org/10.1145/2933267.2933507","url":null,"abstract":"This year's DEBS Grand Challenge offers two very challenging queries over social networks data. These queries -- each for a different reason -- cannot be handled by traditional techniques and therefore call for the development of a specific architecture and data structures. In the first query, the novelty is the non-linearity of the expiration of the elements. Since a traditional sliding window is not suitable, we investigate here the data structures offering the best tradeoffs for all the required operations. In the second query, unlike traditional approaches where no persistent data is stored over the stream, we have to manage a friendship graph which is persistent throughout the system execution. Due to the centrality of this structure, a careful design is therefore required. The common point of the algorithmic approaches that we developed for both queries, is the overwhelming usage of bounds -- upper and lower --, in order execute expensive computations only when required. We devise, for the Query 1, a bound based on the score decay. For the Query 2, we use Turan's theorem to limit the clique computation. The combination of lazy evaluation, careful implementation and thorough testing lead to the realization of an efficient streaming process system.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115146449","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Real-time social network graph analysis using StreamMine3G","authors":"André Martin, Andrey Brito, C. Fetzer","doi":"10.1145/2933267.2933514","DOIUrl":"https://doi.org/10.1145/2933267.2933514","url":null,"abstract":"In this paper, we present our approach for solving the DEBS Grand Challenge 2016 using StreamMine3G, a distributed, highly scalable, elastic and fault tolerant event stream processing (ESP) system. We first provide an overview about StreamMine3G with regards to its programming model and architecture, followed by thorough description of the implementation for the two queries that provide up-to-date information about (i) the top-3 active posts and (ii) the top-k comments with the largest maximum cliques. Novel aspects of our implementation include (i) highly optimized data structures that lower the amount of lookups and traversals, and a (ii) deterministic data partitioning and processing scheme that allows the system to scale without bounds in an elastic fashion while still guaranteeing semantic transparency. In order to better utilize nowadays many-core machines, we furthermore propose a pipelining scheme in addition to data partitioning. Finally, we present a brief performance evaluation of our system.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133809952","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Enrique Saurez, Kirak Hong, D. Lillethun, U. Ramachandran, Beate Ottenwälder
{"title":"Incremental deployment and migration of geo-distributed situation awareness applications in the fog","authors":"Enrique Saurez, Kirak Hong, D. Lillethun, U. Ramachandran, Beate Ottenwälder","doi":"10.1145/2933267.2933317","DOIUrl":"https://doi.org/10.1145/2933267.2933317","url":null,"abstract":"Geo-distributed Situation Awareness applications are large in scale and are characterized by 24/7 data generation from mobile and stationary sensors (such as cameras and GPS devices); latency-sensitivity for converting sensed data to actionable knowledge; and elastic and bursty needs for computational resources. Fog computing [7] envisions providing computational resources close to the edge of the network, consequently reducing the latency for the sense-process-actuate cycle that exists in these applications. We propose Foglets, a programming infrastructure for the geo-distributed computational continuum represented by fog nodes and the cloud. Foglets provides APIs for a spatio-temporal data abstraction for storing and retrieving application generated data on the local nodes, and primitives for communication among the resources in the computational continuum. Foglets manages the application components on the Fog nodes. Algorithms are presented for launching application components and handling the migration of these components between Fog nodes, based on the mobility pattern of the sensors and the dynamic computational needs of the application. Evaluation results are presented for a Fog network consisting of 16 nodes using a simulated vehicular network as the workload. We show that the discovery and deployment protocol can be executed in 0.93 secs, and joining an already deployed application can be as quick as 65 ms. Also, QoS-sensitive proactive migration can be accomplished in 6 ms.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133037800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Automatic learning of predictive rules for complex event processing: doctoral symposium","authors":"Raef Mousheimish, Y. Taher, K. Zeitouni","doi":"10.1145/2933267.2933430","DOIUrl":"https://doi.org/10.1145/2933267.2933430","url":null,"abstract":"The inference mechanisms of CEP engines are completely guided by rules, which are specified manually by domain experts. We argue that this user-based rule specification is a limiting factor, as it requires the experts to have technical knowledge about the CEP language they want to use, it restricts the usage of CEP to merely the detection of straightforward situations, and it restrains its propagation to more advanced fields that require prediction and proactivity. Therefore, we introduce autoCEP as a data mining-based approach that automatically learns predictive CEP rules from historical traces. More precisely, we include our novel method that is capable of learning rules and handling events coming from one source, and then we elaborate our vision on how to extend autoCEP to deal with simultaneous events coming from multiple sources.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"102 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132087519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jags Ramnarayan, Sudhir Menon, S. Wale, Hemant Bhanawat
{"title":"SnappyData: a hybrid system for transactions, analytics, and streaming: demo","authors":"Jags Ramnarayan, Sudhir Menon, S. Wale, Hemant Bhanawat","doi":"10.1145/2933267.2933295","DOIUrl":"https://doi.org/10.1145/2933267.2933295","url":null,"abstract":"An increasing number of applications rely on workflows that involve (1) continuous stream processing, (2) transactional and write-heavy workloads, and (3) interactive SQL analytics. These applications need to consume high-velocity streams to trigger real-time alerts, ingest them into a write-optimized store, and perform OLAP-style analytics to derive deep insight quickly. Consequently, the demand for mixed workloads has resulted in several composite data architectures, exemplified in the \"lambda\" architecture, requiring multiple systems to be stitched together---an exercise that can be hard, time consuming and expensive. Instead, our system, SnappyData, fulfills this promise by (i) enabling streaming, transactions and interactive analytics in a single unifying system---rather than stitching different solutions---and (ii) delivering true interactive speeds via a state-of-the-art approximate query engine that leverages a multitude of synopses as well as the full dataset.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"29 5","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133007338","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Michael Grossniklaus, D. Maier, James Miller, Sharmadha Moorthy, K. Tufte
{"title":"Frames","authors":"Michael Grossniklaus, D. Maier, James Miller, Sharmadha Moorthy, K. Tufte","doi":"10.1145/2933267.2933304","DOIUrl":"https://doi.org/10.1145/2933267.2933304","url":null,"abstract":"Traditional Data Stream Management Systems (DSMS) segment data streams using windows that are defined either by a time interval or a number of tuples. Such windows are fixed---the definition unvarying over the course of a stream---and are defined based on external properties unrelated to the data content of the stream. However, streams and their content do vary over time---the rate of a data stream may vary or the data distribution of the content may vary. The mismatch between a fixed stream segmentation and a variable stream motivates the need for a more flexible, expressive and physically independent stream segmentation. We introduce a new stream segmentation technique, called frames. Frames segment streams based on data content. We present a theory and implementation of frames and show the utility of frames for a variety of applications.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133406139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Experience of event stream processing for top-k queries and dynamic graphs","authors":"Joong-Hyun Choi, Kang-Woo Lee, Eun-Sun Cho","doi":"10.1145/2933267.2933516","DOIUrl":"https://doi.org/10.1145/2933267.2933516","url":null,"abstract":"Solving 2016 ACM DEBS Grand Challenge problems entails both dynamic graph processing and top-k query processing. A straightforward implementation of solutions would not guarantee good performance or prompt responses. This paper shows our experience in implementing solutions of the problems, including rationales of top-k list management techniques we used in our implementation. We also shows the performance evaluation results among three top-k list management schemes and present the reason for our choice for the final result.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115872227","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A policy-based coordination architecture for distributed complex event processing in the internet of things: doctoral symposium","authors":"Jorge C. B. Fonseca, C. Ferraz, Kiev Gama","doi":"10.1145/2933267.2933431","DOIUrl":"https://doi.org/10.1145/2933267.2933431","url":null,"abstract":"The dissemination of powered communication devices has instigated a new technological paradigm called Internet of Things (IoT). These devices are present in our life, capturing information about people's routines and using them as data stream to support many decision-making processes. Nevertheless, many IoT applications take into consideration only the information related to their local context and the data analysis is done by a cloud server, increasing the feedback information latency to the data consumers and reducing the information quality. This work adopts the use of a distributed complex event processing (CEP) to analyse data considering all the IoT devices to execute data processing and distribution through a policy-based coordination architecture, building a μDCEP, called GiTo. The policies define rules for coordination processing which are monitored through the distributed CEP engine. This approach focus on information quality improvement and time reduction between data generation and information acquired by the consumers.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122011835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
César Cañas, Kaiwen Zhang, Bettina Kemme, J. Kienzle, H. Jacobsen
{"title":"Evolving pub/sub subscriptions for multiplayer online games: demo","authors":"César Cañas, Kaiwen Zhang, Bettina Kemme, J. Kienzle, H. Jacobsen","doi":"10.1145/2933267.2933297","DOIUrl":"https://doi.org/10.1145/2933267.2933297","url":null,"abstract":"We identify a class of content-based pub/sub applications with highly dynamic subscriptions. This includes location-based notification systems, predictive stock trading, and multiplayer games. The traditional method of handling subscription changes by engaging in expensive re-subscription protocols is inadequate when the workload is sufficiently large. We propose evolving subscriptions as a technique to alleviate the overhead of subscription churn when handling regular patterns of subscription changes. Subscribers express as a function of time the evolution of its subscriptions. The pub/sub brokers can then independently modify the evolving subscriptions without requiring further communication with the subscribers. In this demo, we present our design and implementation for supporting evolving subscriptions in the context of a multiplayer online game. We first show how evolving subscriptions are expressed to the pub/sub system. We then demonstrate its usage with a game client, which displays real-time performance measurements between the regular system and our enhanced version. We show the relative impact of using evolving subscriptions by running different experiments with our game engine.","PeriodicalId":277061,"journal":{"name":"Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems","volume":"180 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125822602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}