Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)最新文献

{"title":"Data Management Practices on Large-Scale Lustre Scratch File Systems","authors":"Gary L. Rogers, Jesse Hanley, Rick Mohr","doi":"10.1145/2616498.2616545","DOIUrl":"https://doi.org/10.1145/2616498.2616545","url":null,"abstract":"Managing large-scale Lustre scratch file systems is a necessity on shared storage resources. The lack of proper data management can halt all computation that is contingent upon any type of output to disk. Lustre scratch areas are typically provided for users to utilize as high-performance temporary space to stage job input data and store job output data. Common techniques for managing the amount of stored temporary data involve the use of file system quotas or the enforcement of a purge policy that limits how long files can reside in the scratch space. This paper reviews the challenges of balancing usability from the end user's perspective and the administration of these scratch areas, along with the use of tools to decrease the overall amount of time required for file purges on large-scale Lustre systems.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"32 1","pages":"36:1-36:6"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76931562","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fast, Low-Memory Algorithm for Construction of Nanosecond Level Snapshots of Financial Markets","authors":"R. Sinkovits, Tao Feng, Mao Ye","doi":"10.1145/2616498.2616501","DOIUrl":"https://doi.org/10.1145/2616498.2616501","url":null,"abstract":"We present a fast, low-memory algorithm for constructing an order-by-order level snapshot of financial markets with nanosecond resolution. This new implementation is 20-30x faster than an earlier version of the code. In addition, since message data are retained only for as long as it they are needed, the memory footprint is greatly reduced. We find that even the heaviest days of trading spanning the NASDAQ, NYSE and BATS exchanges can now easily be handled using compute nodes with very modest memory (~ 4 GB). A tradeoff of this new approach is that the ability to efficiently manage large numbers of small files is more critical. We demonstrate how we can accommodate these new I/O requirements using the solid-state storage devices (SSDs) on SDSC's Gordon system.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"19 1","pages":"16:1-16:5"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73853629","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"UNICORE in XSEDE: Through Development, Integration, Deployment and beyond","authors":"Tabitha K. Samuel, Shahbaz Memon, Bernd Scheuller, Shava Smallen","doi":"10.1145/2616498.2616554","DOIUrl":"https://doi.org/10.1145/2616498.2616554","url":null,"abstract":"In this paper we discuss the implementation of UNICORE in XSEDE. UNICORE is a Grid middleware tool that was identified by XSEDE to further the areas of remote job submission, campus bridging and workflows. We talk about the overall architecture of UNICORE, a typical HPC environment at XSEDE and why UNICORE is a good fit for this environment. We also discuss the initial efforts made by the UNICORE development team as well as XSEDE's Software development team to integrate UNICORE into the XSEDE landscape. We detail how UNICORE went through the XSEDE engineering process and highlight deployment details at XSEDE. We touch upon how UNICORE is beneficial to the HPC user community. In our final section we talk about future efforts to better integrate UNICORE within XSEDE.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"108 1","pages":"64:1-64:8"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87584080","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"XSEDE Support: Revolutionizing the Next-Generation Therapeutic Drug Discovery","authors":"Bhanu Rekepalli, Y. Peterson","doi":"10.1145/2616498.2616508","DOIUrl":"https://doi.org/10.1145/2616498.2616508","url":null,"abstract":"Drug discovery is a critical but complex and costly endeavor. The rate of approval of new therapeutics by the FDA has been in decline while costs are rising. Increasingly, pharmaceutical companies desire to translate pharmaceutical discovery from academic research in order to decrease risk. Although many researchers have identified very compelling targets, most researchers do not have access to drug discovery resources due to the high cost and complex infrastructure needed to launch a discovery campaign. Long-term objective of this research is to integrate drug interaction simulation software to identify new bioactive molecules and speed drug development with minimum cost and time. This technology is a highly feasible way to rapidly close the therapeutic gap and potentially dramatically improve public health. Initially research was conducted using typical clusters and it took 3 months to perform one run with one conformation of the protein using 1.5 million small molecules. But researchers are interested in working with many proteins with multiple conformations per protein related to entire disease related pathways. At this rate this computational research by itself would take 6 to 7 years of computation on institutional clusters. This resulted in PI applying for the XSEDE allocation with Extended Collaborative Support Services (ECSS) support, which resulted in generation of optimized and scaled the drug interaction workflow on XSEDE supercomputers that reduced computation time for single run from months to 40 minutes using 8000 cores. The results were generated for 5 proteins with 5 conformations with 1.5 million compounds in an afternoon (wall clock time)on Kraken supercomputer which would have taken 5 years of computation on typical cluster. This presentation will discuss about the process from project inception to generating results for publications and proposals for various funding agencies.\u0000 PI quotes \"I thought the computation might not be finished in my life span, this collaboration takes my research to new heights\".","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"10 1","pages":"29:1"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87652051","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Leveraging XSEDE HPC resources to address computational challenges with high-resolution topography data","authors":"C. Youn, V. Nandigam, M. Phan, D. Tarboton, Nancy Wilkins-Diehr, C. Baru, C. Crosby, Anand Padmanabhan, Shaowen Wang","doi":"10.1145/2616498.2616564","DOIUrl":"https://doi.org/10.1145/2616498.2616564","url":null,"abstract":"Leveraging service-oriented architectures and taking advantage of the high-performance compute resources provided by XSEDE, we have developed standards-based web services to address the challenges associated with processing large volumes of high resolution topography data. These web services make results from community software packages and other cyberinfrastructure-based applications available to the wider earth sciences community via the OpenTopography Facility and the CyberGIS Gateway.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"57 1 1","pages":"59:1-59:2"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89064397","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Once you SCOOP, no need to fork","authors":"Yannick Hold-Geoffroy, Olivier Gagnon, M. Parizeau","doi":"10.1145/2616498.2616565","DOIUrl":"https://doi.org/10.1145/2616498.2616565","url":null,"abstract":"This paper presents SCOOP, a new Python framework for automatically distributing dynamic task hierarchies. A task hierarchy refers to tasks that can recursively spawn an arbitrary number of subtasks. The underlying computing infrastructure consists of a simple list of resources. The typical use case is to run the user's main program under the umbrella of the SCOOP module, where it becomes a root task that can spawn any number of subtasks through the standard \"futures\" API of Python, and where these subtasks may themselves spawn other subsubtasks, etc. The full task hierarchy is dynamic in the sense that it is unknown until the end of the last running task. SCOOP automatically distributes tasks amongst available resources using dynamic load balancing. A task is nothing more than a Python callable object in conjunction with its arguments. The user need not worry about message passing implementation details; all communications are implicit.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"38 4","pages":"60:1-60:8"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91440743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An Open Extensible Multi-Target Application Generation Tool for Simple Rapid Deployment of Multi-Scale Scientific Codes","authors":"E. Brookes","doi":"10.1145/2616498.2616560","DOIUrl":"https://doi.org/10.1145/2616498.2616560","url":null,"abstract":"Combining modules wrapping a diversity of executable codes derived from various scientific research labs with a range of computational and data scales into a sustainable framework requires careful considerations. In the described framework, we have separated the module's executable codes from the user-interface and created an application generation tool which produces all the code necessary to create a web based science gateway simultaneously with a local GUI based application. This work was driven by requirements related to an international collaborative grant. This ongoing development is producing applications and will be in the hands of beta testers at the time of this publication.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"191 4","pages":"53:1-53:6"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91479012","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Advanced Application Support for Improved GPU Utilization on Keeneland","authors":"Jeffrey S. Young, M. G. Lopez, Mitchel D. Horton, Richard Glassbrook, J. Vetter","doi":"10.1145/2616498.2616506","DOIUrl":"https://doi.org/10.1145/2616498.2616506","url":null,"abstract":"With the delivery of the Keeneland Full Scale (KFS) system in 2012, XSEDE gained a new, unique GPU computing resource that contains a large number of GPUs per node. In KFS, each node has three NVIDIA Fermi GPUs, for a total of 792 GPUs and a theoretical peak of 614.5 TFLOPS across 264 nodes. While this system provides the potential for extreme productivity, its unique architecture also requires that each user make full use of all the GPU resources on each allocated node to achieve the best performance. Previous publications [12] have demonstrated a tool that allows for tracking the GPU utilization of individual nodes and the system as a whole, and it has helped to pinpoint low GPU utilization numbers on KFS and its precursor KIDS.\u0000 This work discusses experiences, strategies, and results that have been applied on the Keeneland Full Scale system to ensure that users are fully utilizing GPU resources and to improve the performance of their calculations while reducing Service Unit (SU) usage. In many cases, these strategies boil down to two factors: user education and code optimization for KFS's unique architecture. Three specific applications are discussed in this context from the molecular science, materials science, and chemistry domains, and recent application support results are used to illustrate how small interventions can greatly increase utilization on a month-to-month basis.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"20 1","pages":"6:1-6:6"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77394019","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Towards a Scientific Impact Measuring Framework for Large Computing Facilities - a Case Study on XSEDE","authors":"Fugang Wang, G. Laszewski, Geoffrey Fox, T. Furlani, R. L. Deleon, S. Gallo","doi":"10.1145/2616498.2616507","DOIUrl":"https://doi.org/10.1145/2616498.2616507","url":null,"abstract":"We present a framework that (a) integrates publication and citation data retrieval, (b) allows scientific impact metrics generation at different aggregation levels, and (c) provides correlation analysis of impact metrics based on publication and citation data with resource allocation for a computing facility. Furthermore, we use this framework to conduct a scientific impact metrics evaluation of XSEDE. We carry out an extensive statistical analysis correlating XSEDE allocation size to the impact metrics aggregated by project and field of science. This analysis not only helps to provide an indication of XSEDE's scientific impact, but also provides insight regarding maximizing the return on investment in terms of allocation by taking into account the field of science or project based impact metrics. The findings from this analysis can be utilized by the XSEDE resource allocation committee to help assess and identify projects with higher scientific impact. It can also help provide metrics regarding the return on investment for XSEDE resources, or other institutional or campus resources for which an analysis of impact based on publications is important.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"85 1","pages":"25:1-25:8"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80915657","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"XRAS: Allocations software as a service in XSEDE","authors":"David L. Hart, A. Schuele, Ester Soriano, M. Dahan, M. Hanlon","doi":"10.1145/2616498.2616562","DOIUrl":"https://doi.org/10.1145/2616498.2616562","url":null,"abstract":"With a recently deployed software-as-a-service offering, the eXtreme Science and Engineering Discovery Environment (XSEDE) is extending to other client organizations the ability to take advantage of the XSEDE Resource Allocation Service (XRAS), a comprehensive allocations environment for managing the submission, review, and awarding of resource allocations. This effort was launched to refine and revise a legacy system and more than 15 years' worth of incremental improvements; to better integrate and position the allocations service within the XSEDE infrastructure; and to help streamline the allocations processes across a broader set of cyberinfrastructure investments. Because the allocation processes for most high-performance computing providers are typically variations on a common high-level approach, XRAS can deliver allocations software-as-a-service through a set of common tools and interfaces enhanced by flexible mechanisms that allow clients to tailor the environment to meet the needs of their process. By offering such a service to the eScience community, we can leverage the unique expertise of the XSEDE team and expand opportunities for open science by sharing the interfaces needed to manage resource allocations for both large and small institutions.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"114 1","pages":"65:1-65:8"},"PeriodicalIF":0.0,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79245321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}