Proceedings of the 12th Workshop on AI and Scientific Computing at Scale using Flexible Computing Infrastructures最新文献

Large Scale Caching and Streaming of Training Data for Online Deep Learning 面向在线深度学习的大规模缓存和训练数据流

Proceedings of the 12th Workshop on AI and Scientific Computing at Scale using Flexible Computing Infrastructures Pub Date : 2022-06-27 DOI: 10.1145/3526058.3535453

Jie Liu, Bogdan Nicolae, J. Wozniak, Tekin Bicer, Zhengchun Liu

{"title":"Large Scale Caching and Streaming of Training Data for Online Deep Learning","authors":"Jie Liu, Bogdan Nicolae, J. Wozniak, Tekin Bicer, Zhengchun Liu","doi":"10.1145/3526058.3535453","DOIUrl":"https://doi.org/10.1145/3526058.3535453","url":null,"abstract":"The training of deep neural network models on large data remains a difficult problem, despite progress towards scalable techniques. In particular, there is a mismatch between the random but predetermined order in which AI flows select training samples and the streaming I/O patterns for which traditional HPC data storage (e.g., parallel file systems) are designed. In addition, as more data are obtained, it is feasible neither simply to train learning models incrementally, due to catastrophic forgetting (i.e., bias towards new samples), nor to train frequently from scratch, due to prohibitive time and/or resource constraints. In this paper, we study data management techniques that combine caching and streaming with rehearsal support in order to enable efficient access to training samples in both offline training and continual learning. We revisit state-of-art streaming approaches based on data pipelines that transparently handle prefetching, caching, shuffling, and data augmentation, and discuss the challenges and opportunities that arise when combining these methods with data-parallel training techniques. We also report on preliminary experiments that evaluate the I/O overheads involved in accessing the training samples from a parallel file system (PFS) under several concurrency scenarios, highlighting the impact of the PFS on the design of the data pipelines.","PeriodicalId":138397,"journal":{"name":"Proceedings of the 12th Workshop on AI and Scientific Computing at Scale using Flexible Computing Infrastructures","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127271420","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

A Software Ecosystem for Deploying Deep Learning in Gravitational Wave Physics 在引力波物理中部署深度学习的软件生态系统

Proceedings of the 12th Workshop on AI and Scientific Computing at Scale using Flexible Computing Infrastructures Pub Date : 2022-06-27 DOI: 10.1145/3526058.3535454

A. Gunny, D. Rankin, P. Harris, E. Katsavounidis, Ethan Marx, M. Saleem, M. Coughlin, W. Benoit

引用次数: 1

iSeSA

Proceedings of the 12th Workshop on AI and Scientific Computing at Scale using Flexible Computing Infrastructures Pub Date : 2022-06-27 DOI: 10.1145/3526058.3535455

Dheeraj Chahal, Pradeep Gameria, Rajesh Kulkarni, Amit Kalele

{"title":"iSeSA","authors":"Dheeraj Chahal, Pradeep Gameria, Rajesh Kulkarni, Amit Kalele","doi":"10.1145/3526058.3535455","DOIUrl":"https://doi.org/10.1145/3526058.3535455","url":null,"abstract":"High Performance Computing applications typically involve compute intensive simulations which inherently may process large amounts of data at high speeds. The high compute requirements of these applications often exceed the on-premise cluster capacity to analyze the research data. Furthermore, capacity planning becomes very challenging for dynamically varying workloads due to their frequently changing resource demands. These challenges can be mitigated using highly scalable and cost-effective cloud services such as serverless platforms. In this work, we discuss a solution named intelligent Serverless Scalable Architecture framework (iSeSA) that can burst computation to the serverless platform on the public cloud when on-premise compute peak capacity has been reached by providing compute scalability and flexibility. We propose a hybrid cloud model that enables the use of a serverless platform on the public cloud in an automated, intelligent, and cost effective way by applying Machine Learning (ML) and Deep Learning techniques. The iSeSA framework takes a decision to execute the workload on-premise or on serverless platforms. It also manages surge requirements and achieves Service Level Agreement (SLA).","PeriodicalId":138397,"journal":{"name":"Proceedings of the 12th Workshop on AI and Scientific Computing at Scale using Flexible Computing Infrastructures","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116313457","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Proceedings of the 12th Workshop on AI and Scientific Computing at Scale using Flexible Computing Infrastructures 第12届使用灵活计算基础设施的大规模人工智能和科学计算研讨会论文集

Proceedings of the 12th Workshop on AI and Scientific Computing at Scale using Flexible Computing Infrastructures Pub Date : 1900-01-01 DOI: 10.1145/3526058

引用次数: 0