云中的数据密集型超级计算:卫星图像的全球分析

2016 Seventh International Workshop on Data-Intensive Computing in the Clouds (DataCloud) Pub Date : 2016-11-13 DOI:10.1109/DataCloud.2016.7

Michael S. Warren, S. Skillman, R. Chartrand, T. Kelton, R. Keisler, D. Raleigh, M. Turk

{"title":"云中的数据密集型超级计算:卫星图像的全球分析","authors":"Michael S. Warren, S. Skillman, R. Chartrand, T. Kelton, R. Keisler, D. Raleigh, M. Turk","doi":"10.1109/DataCloud.2016.7","DOIUrl":null,"url":null,"abstract":"We present our experiences using cloud computing to support data-intensive analytics on satellite imagery for commercial applications. Drawing from our background in highperformance computing, we draw parallels between the early days of clustered computing systems and the current state of cloud computing and its potential to disrupt the HPC market. Using our own virtual file system layer on top of cloud remote object storage, we demonstrate aggregate read bandwidth of 230 gigabytes per second using 512 Google Compute Engine (GCE) nodes accessing a USA multi-region standard storage bucket. This figure is comparable to the best HPC storage systems in existence. We also present several of our application results, including the identification of field boundaries in Ukraine, and the generation of a global cloud-free base layer from Landsat imagery.","PeriodicalId":325593,"journal":{"name":"2016 Seventh International Workshop on Data-Intensive Computing in the Clouds (DataCloud)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2016-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"11","resultStr":"{\"title\":\"Data-Intensive Supercomputing in the Cloud: Global Analytics for Satellite Imagery\",\"authors\":\"Michael S. Warren, S. Skillman, R. Chartrand, T. Kelton, R. Keisler, D. Raleigh, M. Turk\",\"doi\":\"10.1109/DataCloud.2016.7\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"We present our experiences using cloud computing to support data-intensive analytics on satellite imagery for commercial applications. Drawing from our background in highperformance computing, we draw parallels between the early days of clustered computing systems and the current state of cloud computing and its potential to disrupt the HPC market. Using our own virtual file system layer on top of cloud remote object storage, we demonstrate aggregate read bandwidth of 230 gigabytes per second using 512 Google Compute Engine (GCE) nodes accessing a USA multi-region standard storage bucket. This figure is comparable to the best HPC storage systems in existence. We also present several of our application results, including the identification of field boundaries in Ukraine, and the generation of a global cloud-free base layer from Landsat imagery.\",\"PeriodicalId\":325593,\"journal\":{\"name\":\"2016 Seventh International Workshop on Data-Intensive Computing in the Clouds (DataCloud)\",\"volume\":\"48 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2016-11-13\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"11\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2016 Seventh International Workshop on Data-Intensive Computing in the Clouds (DataCloud)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/DataCloud.2016.7\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2016 Seventh International Workshop on Data-Intensive Computing in the Clouds (DataCloud)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/DataCloud.2016.7","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 11

摘要

我们介绍了我们使用云计算支持商业应用卫星图像数据密集型分析的经验。根据我们在高性能计算方面的背景，我们将早期的集群计算系统与当前的云计算状态及其颠覆高性能计算市场的潜力相提并论。在云远程对象存储之上使用我们自己的虚拟文件系统层，我们演示了使用512个Google计算引擎(GCE)节点访问美国多区域标准存储桶的每秒230千兆字节的总读取带宽。这个数字可以与现有最好的高性能计算存储系统相媲美。我们还介绍了我们的几个应用结果，包括乌克兰野外边界的识别，以及从Landsat图像生成全球无云基础层。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Data-Intensive Supercomputing in the Cloud: Global Analytics for Satellite Imagery

We present our experiences using cloud computing to support data-intensive analytics on satellite imagery for commercial applications. Drawing from our background in highperformance computing, we draw parallels between the early days of clustered computing systems and the current state of cloud computing and its potential to disrupt the HPC market. Using our own virtual file system layer on top of cloud remote object storage, we demonstrate aggregate read bandwidth of 230 gigabytes per second using 512 Google Compute Engine (GCE) nodes accessing a USA multi-region standard storage bucket. This figure is comparable to the best HPC storage systems in existence. We also present several of our application results, including the identification of field boundaries in Ukraine, and the generation of a global cloud-free base layer from Landsat imagery.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2016 Seventh International Workshop on Data-Intensive Computing in the Clouds (DataCloud)

自引率

0.00%

发文量