Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems最新文献

筛选
英文 中文
Travel light: state shedding for efficient operator migration 轻旅行:状态脱落,有效的操作员迁移
Espen Volnes, T. Plagemann, B. Koldehofe, V. Goebel
{"title":"Travel light: state shedding for efficient operator migration","authors":"Espen Volnes, T. Plagemann, B. Koldehofe, V. Goebel","doi":"10.1145/3524860.3539638","DOIUrl":"https://doi.org/10.1145/3524860.3539638","url":null,"abstract":"Operator migration is a crucial concept to adapt event processing systems to dynamic changes. When the placement of a stateful operator changes, the operator state must be migrated to the new host. However, operator state size and time constraints can make it impossible to migrate the operator without severe Quality of Service (QoS) degradation. As a relief, we propose to perform state shedding in such a situation. The core idea of state shedding is to partition the operator state, assign a utility to each partial state, and use the utility and size of each partial state to identify the most useful partial states that can be migrated in a given time frame. Thus, state shedding can maintain a substantially higher QoS with a lower impact on query results than state-of-the-art solutions targeting consistent state at the old and new host. In this paper, we define this novel approach and in a simulation environment evaluate state shedding in migration scenarios with pattern-matching queries.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"62 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115043207","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Toward reducing cross-shard transaction overhead in sharded blockchains 减少分片区块链中的跨分片事务开销
Liuyang Ren, Paul A. S. Ward, B. Wong
{"title":"Toward reducing cross-shard transaction overhead in sharded blockchains","authors":"Liuyang Ren, Paul A. S. Ward, B. Wong","doi":"10.1145/3524860.3539641","DOIUrl":"https://doi.org/10.1145/3524860.3539641","url":null,"abstract":"Sharding is a promising approach to high-performance blockchains and has been extensively explored in academia recently. However, sharding also introduces cross-shard transactions, which require expensive inter-shard coordination to ensure state consistency. Such transactions significantly limit the performance of sharded blockchains. To reduce cross-shard transactions in UTXO-based sharded blockchains, we propose Rooted Graph Placement, which identifies the most appropriate shard for a transaction based on the interaction between the transaction and historical transactions. In conjunction with the placement algorithm, we also devise two techniques to lessen the system performance impact of the remaining cross-shard transactions. One technique parallelizes dependent transaction verification with the atomic commit protocol, and the other consolidates the atomic commit protocol. Combining all the three techniques, we can improve the maximum system throughput by 118% when compared with a state-of-the-art transaction placement algorithm.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"62 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125088260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Detecting trading trends in streaming financial data using Apache Flink 使用Apache Flink检测流金融数据中的交易趋势
Emmanouil Kritharakis, Shengyao Luo, V. Unnikrishnan, Karan Vombatkere
{"title":"Detecting trading trends in streaming financial data using Apache Flink","authors":"Emmanouil Kritharakis, Shengyao Luo, V. Unnikrishnan, Karan Vombatkere","doi":"10.1145/3524860.3539647","DOIUrl":"https://doi.org/10.1145/3524860.3539647","url":null,"abstract":"Modern financial analytics rely on high-volume streams of event notifications that report live market fluctuations based on supply and demand. Accurately identifying trends or breakout patterns based on the Exponential Moving Average (EMA) in the development of an instrument's price early on is an important challenge, so as to buy while the price is low and sell before a downtrend begins. This paper aims to solve the above challenge with a distributed, event-streaming solution built using Apache Flink. We present and implement a solution that leverages customized window operators to calculate the EMA and find breakout patterns, using event generation parallelism to facilitate the rapid processing of the input stream uses sinks to collect and output results, and scales easily on a distributed Flink cluster. We empirically test our design on metrics specified by the benchmarking platform for the DEBS 2022 Grand Challenge and observe a throughput of 45 batches per second and an average latency of 120 ms.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121711113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Window-based parallel operator execution with in-network computing 基于窗口的并行运算执行与网络计算
Bochra Boughzala, Christoph Gärtner, B. Koldehofe
{"title":"Window-based parallel operator execution with in-network computing","authors":"Bochra Boughzala, Christoph Gärtner, B. Koldehofe","doi":"10.1145/3524860.3539804","DOIUrl":"https://doi.org/10.1145/3524860.3539804","url":null,"abstract":"Data parallel processing is a key concept to increase the scalability and elasticity in event streaming systems. Often data parallelism is accomplished in a splitter-merger architecture where the splitter divides incoming streams into partitions and forwards them to parallel operator instances. The splitter performance is a limiting factor to the system throughput and the parallelization degree. This work studies how to leverage novel methods of in-network computing to accelerate the splitter functionality by implementing it as an in-network function. While dedicated hardware for in-network computing has a high potential to enhance the splitter performance, in-network programming models like the P4 language are also highly limited in their expressiveness to support corresponding parallelization models. We propose P4 Splitter Switch (P4SS) which supports overlapping and non-overlapping count-based windows for multiple independent data streams and parallelizes them to a dynamically configurable number of operator instances. We validate in the context of a prototypical implementation our splitting strategy and its scalability in terms of switch resource consumption.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127315483","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Detecting technical trading patterns in financial data with Apache Flink: DEBS grand challenge 2022 使用Apache Flink检测金融数据中的技术交易模式:DEBS大挑战2022
Quan Pham, Quang Nguyen, Ryte Richard, Shekhar Sharma, Xavier Ruiz
{"title":"Detecting technical trading patterns in financial data with Apache Flink: DEBS grand challenge 2022","authors":"Quan Pham, Quang Nguyen, Ryte Richard, Shekhar Sharma, Xavier Ruiz","doi":"10.1145/3524860.3539648","DOIUrl":"https://doi.org/10.1145/3524860.3539648","url":null,"abstract":"The DEBS Grand Challenge is an annual competition in which participants strive to build the fastest and most scalable distributed and event-based systems that solve a practical problem. For the year 2022, the challenge focuses on real-time complex event processing of real-world high-volume financial trading data. The goal is to efficiently compute specific trend indicators and detect patterns that would assist real-life traders in deciding to buy or sell on the financial markets. This paper aims to solve the above challenge using Apache Flink [1] - an open-source, unified stream-processing and batch-processing framework developed by the Apache Software Foundation.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"258 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122660570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Zero-shot cost models for distributed stream processing 分布式流处理的零成本模型
Roman Heinrich, Manisha Luthra, H. Kornmayer, Carsten Binnig
{"title":"Zero-shot cost models for distributed stream processing","authors":"Roman Heinrich, Manisha Luthra, H. Kornmayer, Carsten Binnig","doi":"10.1145/3524860.3539639","DOIUrl":"https://doi.org/10.1145/3524860.3539639","url":null,"abstract":"This paper proposes a learned cost estimation model for Distributed Stream Processing Systems (DSPS) with an aim to provide accurate cost predictions of executing queries. A major premise of this work is that the proposed learned model can generalize to the dynamics of streaming workloads out-of-the-box. This means a model once trained can accurately predict performance metrics such as latency and throughput even if the characteristics of the data and workload or the deployment of operators to hardware changes at runtime. That way the model can be used to solve tasks such as optimizing the placement of operators to minimize the end-to-end latency of a streaming query or maximize its throughput even under varying conditions. Our evaluation on a well-known DSPS, Apache Storm, shows that the model can predict accurately for unseen workloads and queries while generalizing across real-world benchmarks.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"250 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115006189","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
A sneak peek at RisingWave: a cloud-native streaming database RisingWave:一个云原生流媒体数据库
Yanghao Wang, Zhi Liu
{"title":"A sneak peek at RisingWave: a cloud-native streaming database","authors":"Yanghao Wang, Zhi Liu","doi":"10.1145/3524860.3543284","DOIUrl":"https://doi.org/10.1145/3524860.3543284","url":null,"abstract":"This paper presents RisingWave, a new cloud-native streaming database under development. RisingWave's mission is to democratize stream processing: to make stream processing simple, affordable, and accessible. To achieve that, RisingWave treats streams as first-class citizens in a database and uses PostgreSQL compatible SQL interface. Users can create their streaming analytical task by defining materialized views. The views are incrementally maintained in the RisingWave streaming engine, and are ready to be queried directly without exporting to an external system. RisingWave is designed for the cloud. The tiered architecture decouples its components such that each component scales independently to fully leverage the cloud infrastructure and reduce the cost. We demonstrate how RisingWave can simplify users' real-time analytical missions in the modern data stack.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131250188","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Materialize: a platform for building scalable event based systems Materialize:用于构建可伸缩的基于事件的系统的平台
Frank McSherry
{"title":"Materialize: a platform for building scalable event based systems","authors":"Frank McSherry","doi":"10.1145/3524860.3544408","DOIUrl":"https://doi.org/10.1145/3524860.3544408","url":null,"abstract":"Materialize is a system that presents to users as SQL against continually changing data. It transforms inbound streams of *change data capture* events into streams that exactly correspond to transformed data, and maintains indexed representations of the results for efficient access and operation. SQL over changing data is surprisingly (for me) expressive: Materialize can operate on unbounded data, implement data-driven windows, and perform event-based queries, all with ANSI standard SQL. We will discuss what an event-based SQL system looks like, SQL idioms that give rise to traditionally stream-exclusive behavior, and how one architects such a system to scale across multiple dimensions.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133452783","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Substream management in distributed streaming dataflows 分布式流数据流中的子流管理
Artem Trofimov, Nikita Sokolov, Nikita Marshalkin, I. Kuralenok, Boris Novikov
{"title":"Substream management in distributed streaming dataflows","authors":"Artem Trofimov, Nikita Sokolov, Nikita Marshalkin, I. Kuralenok, Boris Novikov","doi":"10.1145/3524860.3539809","DOIUrl":"https://doi.org/10.1145/3524860.3539809","url":null,"abstract":"Most state-of-the-art SPEs use punctuations to divide a stream into bounded substreams of messages, such as epochs and windows. The punctuation approach is powerful but has limitations: it does not support cyclic dataflows, is poorly scalable in some cases due to intensive use of broadcasts, and becomes inefficient when the number of chunks or cluster size becomes significant. We introduce a new substream tracking technique called trAcker that overcomes the limits of punctuations. We experimentally evaluate the properties of trAcker in both synthetic and real-world environments. Experiments show that our technique outperforms punctuations for a large number of substreams and efficiently handles real-world cyclic dataflows.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124432416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
I'm so glad I'm uncoordinated!: coordination is increasingly painful… what can be done? 我很高兴我不协调!:协调工作越来越困难……我们能做些什么?
Pat Helland
{"title":"I'm so glad I'm uncoordinated!: coordination is increasingly painful… what can be done?","authors":"Pat Helland","doi":"10.1145/3524860.3544407","DOIUrl":"https://doi.org/10.1145/3524860.3544407","url":null,"abstract":"In my 44 years building software, technology trends have dramatically changed what's difficult and what's hard. In 1978, CPU, storage, and memory were precious and expensive but coordinating across work was effectively free. Now, we see an abundance of CPU, storage, memory, and bandwidth but we lose a LOT of opportunities for work when we coordinate across parallel activities. This presentation examines the nature of coordination and concludes that it is really aligning disparate arcs of the computations within the work's directed acyclic graph. Parallel systems employ a partial order or DAG (Directed Acyclic Graph) of their work. By aligning these arcs of computation, coordination is \"the partial collapse of partial order\". There are many tricks we use to reduce how often we coordinate and we attempt to taxonomize some of these and explain how they are frequently used to enhance the performance of our systems. We consider the separation of work in space, time, layering, equivalence, and also confluence. Looking at these properties helps us understand the nature of our systems and the implicit or explicit parallelism we use within them. By accepting that coordination is challenging, we can design systems to reduce the pain of these operations. Personally, I am a clumsy and uncoordinated person. I've adapted to this in my personal life by reducing the need to coordinate. Similarly, our computer systems designs work best when designed to be uncoordinated.","PeriodicalId":263415,"journal":{"name":"Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125193842","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书