On-Demand Urgent High Performance Computing Utilizing the Google Cloud Platform

2019 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC) Pub Date : 2019-11-01 DOI:10.1109/UrgentHPC49580.2019.00008

Brandon Posey, Ada E. Deer, Wyatt Gorman, Vanessa July, Neeraj K. Kanhere, D. Speck, Boyd Wilson, A. Apon

{"title":"On-Demand Urgent High Performance Computing Utilizing the Google Cloud Platform","authors":"Brandon Posey, Ada E. Deer, Wyatt Gorman, Vanessa July, Neeraj K. Kanhere, D. Speck, Boyd Wilson, A. Apon","doi":"10.1109/UrgentHPC49580.2019.00008","DOIUrl":null,"url":null,"abstract":"In this paper we describe how high performance computing in the Google Cloud Platform can be utilized in an urgent and emergency situation to process large amounts of traffic data efficiently and on demand. Our approach provides a solution to an urgent need for disaster management using massive data processing and high performance computing. The traffic data used in this demonstration is collected from the public camera systems on Interstate highways in the Southeast United States. Our solution launches a parallel processing system that is the size of a Top 5 supercomputer using the Google Cloud Platform. Results show that the parallel processing system can be launched in a few hours, that it is effective at fast processing of high volume data, and can be de-provisioned in a few hours. We processed 211TB of video utilizing 6,227,593 core hours over the span of about eight hours with an average cost of around $0.008 per vCPU hour, which is less than the cost of many on-premise HPC systems.","PeriodicalId":6723,"journal":{"name":"2019 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","volume":"43 1","pages":"13-23"},"PeriodicalIF":0.0000,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"9","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2019 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/UrgentHPC49580.2019.00008","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 9

Abstract

In this paper we describe how high performance computing in the Google Cloud Platform can be utilized in an urgent and emergency situation to process large amounts of traffic data efficiently and on demand. Our approach provides a solution to an urgent need for disaster management using massive data processing and high performance computing. The traffic data used in this demonstration is collected from the public camera systems on Interstate highways in the Southeast United States. Our solution launches a parallel processing system that is the size of a Top 5 supercomputer using the Google Cloud Platform. Results show that the parallel processing system can be launched in a few hours, that it is effective at fast processing of high volume data, and can be de-provisioned in a few hours. We processed 211TB of video utilizing 6,227,593 core hours over the span of about eight hours with an average cost of around $0.008 per vCPU hour, which is less than the cost of many on-premise HPC systems.

查看原文本刊更多论文

利用谷歌云平台的按需紧急高性能计算

在本文中，我们描述了如何在紧急和紧急情况下利用谷歌云平台中的高性能计算来高效和按需处理大量交通数据。我们的方法为使用大规模数据处理和高性能计算的灾难管理的迫切需求提供了解决方案。本演示中使用的交通数据是从美国东南部州际公路上的公共摄像系统收集的。我们的解决方案启动了一个并行处理系统，其大小相当于使用谷歌云平台的Top 5超级计算机。结果表明，该并行处理系统可以在几个小时内启动，对大容量数据的快速处理是有效的，并且可以在几个小时内解除预置。我们在大约8小时的时间内使用6,227,593个核心小时处理了211TB的视频，平均成本约为每vCPU小时0.008美元，这比许多本地HPC系统的成本要低。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2019 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC)

自引率

0.00%

发文量