D0 SAM系统中的分布式数据访问和资源管理

Proceedings 10th IEEE International Symposium on High Performance Distributed Computing Pub Date : 2001-08-07 DOI:10.1109/HPDC.2001.945179

I. Terekhov, R. Pordes, V. White, L. Lueking, L. Loebel-Carpenter, J. Trumbo, S. Veseli, M. Vranicar, S. White, H. Schellman

{"title":"D0 SAM系统中的分布式数据访问和资源管理","authors":"I. Terekhov, R. Pordes, V. White, L. Lueking, L. Loebel-Carpenter, J. Trumbo, S. Veseli, M. Vranicar, S. White, H. Schellman","doi":"10.1109/HPDC.2001.945179","DOIUrl":null,"url":null,"abstract":"SAM (Sequential Access through Meta-data) is the data access and job management system for the D0 high energy physics experiment at Fermilab. The SAM system is being developed and used to handle the Petabyte-scale experiment data, accessed by hundreds of D0 collaborators scattered around the world. In this paper, we present solutions to some of the distributed data processing problems from the perspective of real experience dealing with mission-critical data. We concentrate on the distributed disk caching, resource management and job control. The system has elements of Grid computing and has features applicable to data-intensive computing in general.","PeriodicalId":304683,"journal":{"name":"Proceedings 10th IEEE International Symposium on High Performance Distributed Computing","volume":"46 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2001-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"32","resultStr":"{\"title\":\"Distributed data access and resource management in the D0 SAM system\",\"authors\":\"I. Terekhov, R. Pordes, V. White, L. Lueking, L. Loebel-Carpenter, J. Trumbo, S. Veseli, M. Vranicar, S. White, H. Schellman\",\"doi\":\"10.1109/HPDC.2001.945179\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"SAM (Sequential Access through Meta-data) is the data access and job management system for the D0 high energy physics experiment at Fermilab. The SAM system is being developed and used to handle the Petabyte-scale experiment data, accessed by hundreds of D0 collaborators scattered around the world. In this paper, we present solutions to some of the distributed data processing problems from the perspective of real experience dealing with mission-critical data. We concentrate on the distributed disk caching, resource management and job control. The system has elements of Grid computing and has features applicable to data-intensive computing in general.\",\"PeriodicalId\":304683,\"journal\":{\"name\":\"Proceedings 10th IEEE International Symposium on High Performance Distributed Computing\",\"volume\":\"46 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2001-08-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"32\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings 10th IEEE International Symposium on High Performance Distributed Computing\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/HPDC.2001.945179\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings 10th IEEE International Symposium on High Performance Distributed Computing","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/HPDC.2001.945179","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 32

摘要

SAM (Sequential Access through Meta-data)是Fermilab D0高能物理实验的数据访问和作业管理系统。SAM系统正在开发中，并用于处理分布在世界各地的数百名D0合作者访问的pb级实验数据。在本文中，我们从处理关键任务数据的实际经验的角度提出了一些分布式数据处理问题的解决方案。我们专注于分布式磁盘缓存、资源管理和作业控制。该系统具有网格计算的元素，具有适用于一般数据密集型计算的特性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Distributed data access and resource management in the D0 SAM system

SAM (Sequential Access through Meta-data) is the data access and job management system for the D0 high energy physics experiment at Fermilab. The SAM system is being developed and used to handle the Petabyte-scale experiment data, accessed by hundreds of D0 collaborators scattered around the world. In this paper, we present solutions to some of the distributed data processing problems from the perspective of real experience dealing with mission-critical data. We concentrate on the distributed disk caching, resource management and job control. The system has elements of Grid computing and has features applicable to data-intensive computing in general.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Proceedings 10th IEEE International Symposium on High Performance Distributed Computing

自引率

0.00%

发文量