Processing and Analytics of Big Data Streams with Yahoo!S4

2015 IEEE 29th International Conference on Advanced Information Networking and Applications Workshops Pub Date : 2015-03-24 DOI:10.1109/AINA.2015.194

F. Xhafa, V. Naranjo, S. Caballé

{"title":"Processing and Analytics of Big Data Streams with Yahoo!S4","authors":"F. Xhafa, V. Naranjo, S. Caballé","doi":"10.1109/AINA.2015.194","DOIUrl":null,"url":null,"abstract":"Many Internet-based applications generate huge data streams, which are known as Big Data Streams. Such applications comprise IoT-based monitoring systems, data analytics from monitoring online learning workspaces and MOOCs, global flight monitoring systems, etc. Differently from Big Data processing in which the data is available in databases, file systems, etc., before processing, in Big Data Streams the data stream is unbounded and it is to be processed as it becomes available. Besides the challenges of processing huge amount of data, the Big Data Stream processing adds further challenges of coping with scalability and high throughput to enable real time decision taking. While for Big Data processing the MapReduce framework has resulted successful, its batch mode processing shows limitations to process Big Data Streams. Therefore there have been proposed alternative frameworks such as Yahoo!S4, Twitter Storm, etc., to Big Data Stream processing. In this paper we implement and evaluate the Yahoo!S4 for Big Data Stream processing and exemplify through the Big Data Stream from global flight monitoring system.","PeriodicalId":6845,"journal":{"name":"2015 IEEE 29th International Conference on Advanced Information Networking and Applications Workshops","volume":"51 1","pages":"263-270"},"PeriodicalIF":0.0000,"publicationDate":"2015-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"34","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2015 IEEE 29th International Conference on Advanced Information Networking and Applications Workshops","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/AINA.2015.194","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 34

Abstract

Many Internet-based applications generate huge data streams, which are known as Big Data Streams. Such applications comprise IoT-based monitoring systems, data analytics from monitoring online learning workspaces and MOOCs, global flight monitoring systems, etc. Differently from Big Data processing in which the data is available in databases, file systems, etc., before processing, in Big Data Streams the data stream is unbounded and it is to be processed as it becomes available. Besides the challenges of processing huge amount of data, the Big Data Stream processing adds further challenges of coping with scalability and high throughput to enable real time decision taking. While for Big Data processing the MapReduce framework has resulted successful, its batch mode processing shows limitations to process Big Data Streams. Therefore there have been proposed alternative frameworks such as Yahoo!S4, Twitter Storm, etc., to Big Data Stream processing. In this paper we implement and evaluate the Yahoo!S4 for Big Data Stream processing and exemplify through the Big Data Stream from global flight monitoring system.

查看原文本刊更多论文

雅虎大数据流处理与分析S4

许多基于互联网的应用程序产生巨大的数据流，这被称为大数据流。这些应用包括基于物联网的监测系统、监测在线学习工作空间和mooc的数据分析、全球飞行监测系统等。与大数据处理不同的是，在处理之前，数据在数据库、文件系统等中是可用的，而在大数据流中，数据流是无限的，当数据可用时才进行处理。除了处理大量数据的挑战之外，大数据流处理还增加了应对可扩展性和高吞吐量的挑战，以实现实时决策。虽然MapReduce框架在处理大数据方面取得了成功，但它的批处理模式在处理大数据流方面存在局限性。因此，有人提出了替代框架，如Yahoo!S4、Twitter Storm等，对大数据流进行处理。本文对Yahoo!S4用于大数据流处理，并通过来自全球飞行监控系统的大数据流举例说明。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2015 IEEE 29th International Conference on Advanced Information Networking and Applications Workshops

自引率

0.00%

发文量