软件开发框架为分布式存储和GPGPU数据处理基础设施

2016 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (EIConRusNW) Pub Date : 2016-02-01 DOI:10.1109/EICONRUSNW.2016.7448158

I. S. Kamenskikh, D. M. Sinelnikov, D. S. Kalintsev, A. Kozlov, M. M. Rovnyagin, D. Shulga

{"title":"软件开发框架为分布式存储和GPGPU数据处理基础设施","authors":"I. S. Kamenskikh, D. M. Sinelnikov, D. S. Kalintsev, A. Kozlov, M. M. Rovnyagin, D. Shulga","doi":"10.1109/EICONRUSNW.2016.7448158","DOIUrl":null,"url":null,"abstract":"The problem of choosing the cluster or a cluster node for task execution is important for the overall performance of a distributed system. This paper presents a complex approach to the planning of computations on heterogeneous distributed systems - a set of clusters and NoSQL storage systems. Dynamic scheduling algorithm depends on: the inter-cluster network parameters, characteristics of cluster interconnect, compute nodes utilization, co-processors computing capabilities, etc. In this work Hadoop YARN, CUDA technology and NoSQL-system Apache Cassandra has been used as the experimental platform.","PeriodicalId":262452,"journal":{"name":"2016 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (EIConRusNW)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2016-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"5","resultStr":"{\"title\":\"Software development framework for a distributed storage and GPGPU data processing infrastructure\",\"authors\":\"I. S. Kamenskikh, D. M. Sinelnikov, D. S. Kalintsev, A. Kozlov, M. M. Rovnyagin, D. Shulga\",\"doi\":\"10.1109/EICONRUSNW.2016.7448158\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The problem of choosing the cluster or a cluster node for task execution is important for the overall performance of a distributed system. This paper presents a complex approach to the planning of computations on heterogeneous distributed systems - a set of clusters and NoSQL storage systems. Dynamic scheduling algorithm depends on: the inter-cluster network parameters, characteristics of cluster interconnect, compute nodes utilization, co-processors computing capabilities, etc. In this work Hadoop YARN, CUDA technology and NoSQL-system Apache Cassandra has been used as the experimental platform.\",\"PeriodicalId\":262452,\"journal\":{\"name\":\"2016 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (EIConRusNW)\",\"volume\":\"42 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2016-02-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"5\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2016 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (EIConRusNW)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/EICONRUSNW.2016.7448158\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2016 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (EIConRusNW)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/EICONRUSNW.2016.7448158","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 5

摘要

选择集群或集群节点执行任务的问题对于分布式系统的整体性能非常重要。本文提出了一种复杂的方法来规划异构分布式系统上的计算——一组集群和NoSQL存储系统。动态调度算法取决于:集群间网络参数、集群互联特性、计算节点利用率、协处理器计算能力等。本工作采用Hadoop YARN、CUDA技术和nosql系统Apache Cassandra作为实验平台。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Software development framework for a distributed storage and GPGPU data processing infrastructure

The problem of choosing the cluster or a cluster node for task execution is important for the overall performance of a distributed system. This paper presents a complex approach to the planning of computations on heterogeneous distributed systems - a set of clusters and NoSQL storage systems. Dynamic scheduling algorithm depends on: the inter-cluster network parameters, characteristics of cluster interconnect, compute nodes utilization, co-processors computing capabilities, etc. In this work Hadoop YARN, CUDA technology and NoSQL-system Apache Cassandra has been used as the experimental platform.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2016 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (EIConRusNW)

自引率

0.00%

发文量