使用本地缓存系统提高HDFS的性能

Second International Conference on Future Generation Communication Technologies (FGCT 2013) Pub Date : 2013-11-01 DOI:10.1109/FGCT.2013.6767200

Sangwon Yoon, In-Yong Jung, Ki-Hyun Kim, C. Jeong

{"title":"使用本地缓存系统提高HDFS的性能","authors":"Sangwon Yoon, In-Yong Jung, Ki-Hyun Kim, C. Jeong","doi":"10.1109/FGCT.2013.6767200","DOIUrl":null,"url":null,"abstract":"I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.","PeriodicalId":200083,"journal":{"name":"Second International Conference on Future Generation Communication Technologies (FGCT 2013)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"5","resultStr":"{\"title\":\"Improving HDFS performance using local caching system\",\"authors\":\"Sangwon Yoon, In-Yong Jung, Ki-Hyun Kim, C. Jeong\",\"doi\":\"10.1109/FGCT.2013.6767200\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.\",\"PeriodicalId\":200083,\"journal\":{\"name\":\"Second International Conference on Future Generation Communication Technologies (FGCT 2013)\",\"volume\":\"46 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2013-11-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"5\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Second International Conference on Future Generation Communication Technologies (FGCT 2013)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/FGCT.2013.6767200\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Second International Conference on Future Generation Communication Technologies (FGCT 2013)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/FGCT.2013.6767200","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 5

摘要

在各种分布式并行集群中，文件系统的I/O性能会严重影响内容感知应用程序的性能。本文提出了一种新的本地缓存系统架构，通过使用合适的缓存算法和管理内容队列和位置来调整本地文件系统和Hadoop分布式文件系统之间的内容位置。此外，我们还展示了我们的本地缓存系统可以显著提高内容感知应用程序的性能。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Improving HDFS performance using local caching system

I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Second International Conference on Future Generation Communication Technologies (FGCT 2013)

自引率

0.00%

发文量