{"title":"基于HDFS的基于tile的可扩展栅格数据管理系统","authors":"Guangqing Zhang, Chuanjie Xie, Lei Shi, Yunyan Du","doi":"10.1109/Geoinformatics.2012.6270280","DOIUrl":null,"url":null,"abstract":"Hadoop has become a worldwide popular open source platform for large data analysis in commercial application and Hadoop distributed file system (HDFS) is the core part of it. However, HDFS cannot be used directly for managing raster data, for the geographic location information is involved. In this paper, we describe the implementation of a tile-based scalable raster data management system based on HDFS. While reserving the basic architecture of HDFS, we reorganize the data structure in block, add some additional metadata, design an index data structure in block, keep an overlapping region between adjacent blocks, and offer a compression option for users. Besides, we provide functions for reading the raster data from HDFS in tile stream. These optimizations match the feature of raster data to the architecture of HDFS. MapReduce Applications can be built on the raster data management system.","PeriodicalId":259976,"journal":{"name":"2012 20th International Conference on Geoinformatics","volume":"49 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2012-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"9","resultStr":"{\"title\":\"A tile-based scalable raster data management system based on HDFS\",\"authors\":\"Guangqing Zhang, Chuanjie Xie, Lei Shi, Yunyan Du\",\"doi\":\"10.1109/Geoinformatics.2012.6270280\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Hadoop has become a worldwide popular open source platform for large data analysis in commercial application and Hadoop distributed file system (HDFS) is the core part of it. However, HDFS cannot be used directly for managing raster data, for the geographic location information is involved. In this paper, we describe the implementation of a tile-based scalable raster data management system based on HDFS. While reserving the basic architecture of HDFS, we reorganize the data structure in block, add some additional metadata, design an index data structure in block, keep an overlapping region between adjacent blocks, and offer a compression option for users. Besides, we provide functions for reading the raster data from HDFS in tile stream. These optimizations match the feature of raster data to the architecture of HDFS. MapReduce Applications can be built on the raster data management system.\",\"PeriodicalId\":259976,\"journal\":{\"name\":\"2012 20th International Conference on Geoinformatics\",\"volume\":\"49 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2012-06-15\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"9\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2012 20th International Conference on Geoinformatics\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/Geoinformatics.2012.6270280\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2012 20th International Conference on Geoinformatics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/Geoinformatics.2012.6270280","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
A tile-based scalable raster data management system based on HDFS
Hadoop has become a worldwide popular open source platform for large data analysis in commercial application and Hadoop distributed file system (HDFS) is the core part of it. However, HDFS cannot be used directly for managing raster data, for the geographic location information is involved. In this paper, we describe the implementation of a tile-based scalable raster data management system based on HDFS. While reserving the basic architecture of HDFS, we reorganize the data structure in block, add some additional metadata, design an index data structure in block, keep an overlapping region between adjacent blocks, and offer a compression option for users. Besides, we provide functions for reading the raster data from HDFS in tile stream. These optimizations match the feature of raster data to the architecture of HDFS. MapReduce Applications can be built on the raster data management system.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
