Ole Meyer, Julius Ollesch, Stefan Gries, F. Wessling, V. Gruhn
{"title":"Property-based routing in clustered message brokers for CPS: Doctoral Symposium","authors":"Ole Meyer, Julius Ollesch, Stefan Gries, F. Wessling, V. Gruhn","doi":"10.1145/3093742.3096475","DOIUrl":"https://doi.org/10.1145/3093742.3096475","url":null,"abstract":"Cyber-Physical Systems (CPS) are interconnected systems that can measure, manipulate, and adapt their environment via sensors and actors. The high number of measured data means that a reliable and scalable communication infrastructure is indispensable, especially if data is processed in real time. Data can be available in different measurement qualities, which usability depends on the particular application. As a result, data is regularly discarded, resulting in network inefficiencies when they are previously transmitted. This effect becomes more important as the number of heterogeneous sensors increases. In this paper, we discuss the implications and show our first approach to solve the problem based on MQTT [1], one of the most widely used public-subscribe protocols in the area of IoT and CPS.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116676871","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"FlinkMan: Anomaly Detection in Manufacturing Equipment with Apache Flink: Grand Challenge","authors":"Nicolo Rivetti, Yann Busnel, A. Gal","doi":"10.1145/3093742.3095099","DOIUrl":"https://doi.org/10.1145/3093742.3095099","url":null,"abstract":"We present a (soft) real-time event-based anomaly detection application for manufacturing equipment, built on top of the general purpose stream processing framework Apache Flink. The anomaly detection involves multiple CPUs and/or memory intensive tasks, such as clustering on large time-based window and parsing input data in RDF-format. The main goal is to reduce end-to-end latencies, while handling high input throughput and still provide exact results. Given a truly distributed setting, this challenge also entails careful task and/or data parallelization and balancing. We propose FlinkMan, a system that offers a generic and efficient solution, which maximizes the usage of available cores and balances the load among them. We illustrates the accuracy and efficiency of FlinkMan, over a 3-step pipelined data stream analysis, that includes clustering, modeling and querying.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114508640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dimitrije Jankov, Sourav Sikdar, Rohan Mukherjee, Kia Teymourian, C. Jermaine
{"title":"Real-time High Performance Anomaly Detection over Data Streams: Grand Challenge","authors":"Dimitrije Jankov, Sourav Sikdar, Rohan Mukherjee, Kia Teymourian, C. Jermaine","doi":"10.1145/3093742.3095102","DOIUrl":"https://doi.org/10.1145/3093742.3095102","url":null,"abstract":"Real-time analytics over data streams are crucial for a wide range of use cases in industry and research. Today's sensor systems can produce high throughput data streams that have to be analyzed in real-time. One important analytic task is anomaly or outlier detection from the streaming data. In many industry applications, sensing devices produce a data stream that can be monitored to know the correct operation of industry devices and consequently avoid damages by triggering reactions in real-time. While anomaly detection is a well-studied topic in data mining, the real-time high-performance anomaly detection from big data streams require special studies and well-optimized implementation. This paper presents our implementation of a real-time anomaly detection system over data streams. We outline details of our two separate implementations using the Java and C++ programming languages, and provide technical details about the data processing pipelines. We report experimental results and describe performance tuning strategies.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130144735","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reflections on Almost Two Decades of Research into Stream Processing: Tutorial","authors":"K. S. Esmaili","doi":"10.1145/3093742.3095110","DOIUrl":"https://doi.org/10.1145/3093742.3095110","url":null,"abstract":"Ever since the need for new approaches and systems to handle data streams was identified in early 2000s, stream processing has been an active area of research, resulting in a large body of work with significant impact. This tutorial reflects on this research history by highlighting a number of trends and best practices that can be identified in hindsight. It also enumerates a list of directions for future research in stream processing.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"235 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120880310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Blockchain: Distributed Event-based Processing in a Data-Centric World: Extended Abstract","authors":"R. Hull","doi":"10.1145/3093742.3097982","DOIUrl":"https://doi.org/10.1145/3093742.3097982","url":null,"abstract":"Usage of Blockchain is expanding from the initial focus on crypto-currencies towards applications to support a broad range of collaborative activies amongst businesses, organizations, and individuals. There are two broad levels of Blockchain: the foundation level relates to encryption, consensus algorithms, and support for a single (logical) data store that is shared by all participants; and the \"smart contract\" level that enables developers and business-level users to specify the data, logic, and behavior that collaborations should follow. The smart contracts are programs that are fundamentally event driven, data-centric, and support the activity of a distributed set of stakeholders situated across multiple organizations. This raises an array of research challenges in areas including language and solution design, interoperation across smart contracts, and verification.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124573641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Requirements Engineering Perspective on Events in Cyber-Physical Systems: Poster","authors":"Julius Ollesch, M. Hesenius, V. Gruhn, C. Alias","doi":"10.1145/3093742.3095097","DOIUrl":"https://doi.org/10.1145/3093742.3095097","url":null,"abstract":"Cyber-physical systems (CPS) require event-based control paradigms such as complex event processing (CEP) to have adaptive analytical control mechanisms. CPS are becoming a high-profile research topic as they are key to disruptive digital innovations. We discuss from a requirements engineering perspective how current approaches lack support for the engineering of CPS events and respective CEP solutions. In addition, the paper summarizes results of a comparison of event models and preliminary work on a logistics case study.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132483503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"StreamLearner: Distributed Incremental Machine Learning on Event Streams: Grand Challenge","authors":"C. Mayer, R. Mayer, M. Abdo","doi":"10.1145/3093742.3095103","DOIUrl":"https://doi.org/10.1145/3093742.3095103","url":null,"abstract":"Today, massive amounts of streaming data from smart devices need to be analyzed automatically to realize the Internet of Things. The Complex Event Processing (CEP) paradigm promises low-latency pattern detection on event streams. However, CEP systems need to be extended with Machine Learning (ML) capabilities such as online training and inference in order to be able to detect fuzzy patterns (e.g. outliers) and to improve pattern recognition accuracy during runtime using incremental model training. In this paper, we propose a distributed CEP system denoted as StreamLearner for ML-enabled complex event detection. The proposed programming model and data-parallel system architecture enable a wide range of real-world applications and allow for dynamically scaling up and out system resources for low-latency, high-throughput event processing. We show that the DEBS Grand Challenge 2017 case study (i.e., anomaly detection in smart factories) integrates seamlessly into the StreamLearner API. Our experiments verify scalability and high event throughput of StreamLearner.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130435446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jürgen Thanhofer-Pilisch, Rick Rabiser, Thomas Krismayer, Michael Vierhauser, P. Grünbacher, Stefan Wallner, Klaus Seyerlehner, H. Zeisel
{"title":"An Event-based Capture-and-Compare Approach to Support the Evolution of Systems of Systems","authors":"Jürgen Thanhofer-Pilisch, Rick Rabiser, Thomas Krismayer, Michael Vierhauser, P. Grünbacher, Stefan Wallner, Klaus Seyerlehner, H. Zeisel","doi":"10.1145/3093742.3093909","DOIUrl":"https://doi.org/10.1145/3093742.3093909","url":null,"abstract":"Industrial software systems are often systems of systems (SoS) that evolve continuously to meet new customer requirements or to address technological changes. Despite thorough testing of the different contributing parts, the full behavior of SoS only emerges at runtime. The systems in the SoS and their interactions thus need to be continuously monitored and checked during operation to determine compliance with requirements. In particular, after changes to one system, it is necessary to check whether the overall SoS still behaves correctly and as intended. Based on an existing monitoring framework we have been developing support for capturing and comparing event traces in SoS. Our approach facilitates and partly automates the identification of differences in event traces, which often indicate undesirable behavior introduced during evolution. In this paper we motivate the need for monitoring and evolution support in SoS using an industrial example and describe our event-based capture-and-compare approach. We evaluate the applicability and scalability of our tool-supported approach, demonstrating that it can cope with comparing event traces from an industrial SoS. We present our experiences and findings intended for researchers and practitioners working on maintenance and evolution of large-scale software systems.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129090947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Kafka versus RabbitMQ: A comparative study of two industry reference publish/subscribe implementations: Industry Paper","authors":"P. Dobbelaere, K. S. Esmaili","doi":"10.1145/3093742.3093908","DOIUrl":"https://doi.org/10.1145/3093742.3093908","url":null,"abstract":"Publish/subscribe is a distributed interaction paradigm well adapted to the deployment of scalable and loosely coupled systems. Apache Kafka and RabbitMQ are two popular open-source and commercially-supported pub/sub systems that have been around for almost a decade and have seen wide adoption. Given the popularity of these two systems and the fact that both are branded as pub/sub systems, two frequently asked questions in the relevant online forums are: how do they compare against each other and which one to use? In this paper, we frame the arguments in a holistic approach by establishing a common comparison framework based on the core functionalities of pub/sub systems. Using this framework, we then venture into a qualitative and quantitative (i.e. empirical) comparison of the common features of the two systems. Additionally, we also highlight the distinct features that each of these systems has. After enumerating a set of use cases that are best suited for RabbitMQ or Kafka, we try to guide the reader through a determination table to choose the best architecture given his/her particular set of requirements.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"129 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117212963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Guenter Hesse, Christoph Matthies, Benjamin Reissaus, M. Uflacker
{"title":"A New Application Benchmark for Data Stream Processing Architectures in an Enterprise Context: Doctoral Symposium","authors":"Guenter Hesse, Christoph Matthies, Benjamin Reissaus, M. Uflacker","doi":"10.1145/3093742.3093902","DOIUrl":"https://doi.org/10.1145/3093742.3093902","url":null,"abstract":"Against the backdrop of ever-growing data volumes and trends like the Internet of Things (IoT) or Industry 4.0, Data Stream Processing Systems (DSPSs) or data stream processing architectures in general receive a greater interest. Continuously analyzing streams of data allows immediate responses to environmental changes. A challenging task in that context is assessing and comparing data stream processing architectures in order to identify the most suitable one for certain settings. The present paper provides an overview about performance benchmarks that can be used for analyzing data stream processing applications. By describing shortcomings of these benchmarks, the need for a new application benchmark in this area, especially for a benchmark covering enterprise architectures, is highlighted. A key role in such an enterprise context is the combination of streaming data and business data, which is barely covered in current data stream processing benchmarks. Furthermore, first ideas towards the development of a solution, i.e., a new application benchmark that is able to fill the existing gap, are depicted.","PeriodicalId":325666,"journal":{"name":"Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130884436","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}