女皇:为极端规模的科学模拟提供可扩展的元数据

Proceedings of the 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems Pub Date : 2017-11-12 DOI:10.1145/3149393.3149403

Margaret Lawson, C. Ulmer, Shyamali Mukherjee, G. Templet, J. Lofstead, Scott Levy, Patrick M. Widener, T. Kordenbrock

{"title":"女皇:为极端规模的科学模拟提供可扩展的元数据","authors":"Margaret Lawson, C. Ulmer, Shyamali Mukherjee, G. Templet, J. Lofstead, Scott Levy, Patrick M. Widener, T. Kordenbrock","doi":"10.1145/3149393.3149403","DOIUrl":null,"url":null,"abstract":"Significant challenges exist in the efficient retrieval of data from extreme-scale simulations. An important and evolving method of addressing these challenges is application-level metadata management. Historically, HDF5 and NetCDF have eased data retrieval by offering rudimentary attribute capabilities that provide basic metadata. ADIOS simplified data retrieval by utilizing metadata for each process' data. EMPRESS provides a simple example of the next step in this evolution by integrating per-process metadata with the storage system itself, making it more broadly useful than single file or application formats. Additionally, it allows for more robust and customizable metadata.","PeriodicalId":262458,"journal":{"name":"Proceedings of the 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems","volume":"60 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"14","resultStr":"{\"title\":\"Empress: extensible metadata provider for extreme-scale scientific simulations\",\"authors\":\"Margaret Lawson, C. Ulmer, Shyamali Mukherjee, G. Templet, J. Lofstead, Scott Levy, Patrick M. Widener, T. Kordenbrock\",\"doi\":\"10.1145/3149393.3149403\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Significant challenges exist in the efficient retrieval of data from extreme-scale simulations. An important and evolving method of addressing these challenges is application-level metadata management. Historically, HDF5 and NetCDF have eased data retrieval by offering rudimentary attribute capabilities that provide basic metadata. ADIOS simplified data retrieval by utilizing metadata for each process' data. EMPRESS provides a simple example of the next step in this evolution by integrating per-process metadata with the storage system itself, making it more broadly useful than single file or application formats. Additionally, it allows for more robust and customizable metadata.\",\"PeriodicalId\":262458,\"journal\":{\"name\":\"Proceedings of the 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems\",\"volume\":\"60 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2017-11-12\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"14\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3149393.3149403\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3149393.3149403","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 14

摘要

从极端尺度模拟中有效检索数据存在重大挑战。解决这些挑战的一个重要且不断发展的方法是应用程序级元数据管理。从历史上看，HDF5和NetCDF通过提供提供基本元数据的基本属性功能来简化数据检索。ADIOS通过利用每个过程数据的元数据简化了数据检索。EMPRESS通过将每进程元数据与存储系统本身集成在一起，使其比单一文件或应用程序格式更广泛地有用，从而提供了这一发展的下一步的简单示例。此外，它支持更健壮和可定制的元数据。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Empress: extensible metadata provider for extreme-scale scientific simulations

Significant challenges exist in the efficient retrieval of data from extreme-scale simulations. An important and evolving method of addressing these challenges is application-level metadata management. Historically, HDF5 and NetCDF have eased data retrieval by offering rudimentary attribute capabilities that provide basic metadata. ADIOS simplified data retrieval by utilizing metadata for each process' data. EMPRESS provides a simple example of the next step in this evolution by integrating per-process metadata with the storage system itself, making it more broadly useful than single file or application formats. Additionally, it allows for more robust and customizable metadata.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Proceedings of the 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems

自引率

0.00%

发文量