{"title":"Instrumentation for Accurate Energy-to-Solution Measurements of a Texas Instruments TMS320C6678 Digital Signal Processor and Its DDR3 Memory","authors":"G. Netzer, S. Johnsson, Daniel Ahlin, E. Laure","doi":"10.1109/E2SC.2014.12","DOIUrl":"https://doi.org/10.1109/E2SC.2014.12","url":null,"abstract":"Architectural choices for High-Performance Computing systems have once again become interesting with energy efficiency for targeted workloads now being a major decision factor. A detailed understanding of the energy consumption of major system components during code execution is critical for evolving architectures towards enhanced energy efficiency. The focus of this paper is on the measurement system hard-and software we designed and implemented for the assessment of the energy-to-solution of HPC workloads for the Texas Instruments TMS320C6678 (6678) Digital Signal Processor. The 6678's thermal design power falls between x86 server processors and mobile CPUs and so does its floating-point and memory system capabilities. Yet, compared to those types of processors in corresponding CMOS technology, it offers a potentially significant energy advantage. The measurement system is described together with a thorough error analysis. Measurements are processed out-of-band minimizing the impact on the measured system. Sample observations of the energy efficiency of the 6678 and its memory system are included for illustration.","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134520017","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating Performance and Power Efficiency of Scientific Applications on Multi-threaded Systems","authors":"R. Gioiosa, D. Kerbyson, A. Hoisie","doi":"10.1109/E2SC.2014.15","DOIUrl":"https://doi.org/10.1109/E2SC.2014.15","url":null,"abstract":"The power and energy walls are changing the way users utilize supercomputers: Time-to-completion is not the only important goal but other metrics, such as the energy required to solve a problem or the power efficiency, are becoming as important as performance. This shift towards power- and energy-aware computing is expected to continue in the exascale era, thus, understanding the performance, power and energy implications of different hardware configurations is of paramount importance. In this paper we analyze the performance, power efficiency and energy consumption of scientific applications programmed in MPI, OpenMP and MPI+OpenMP on two different architectures that have take different approaches to limit power consumption, IBM POWER7+ and AMD Interlagos. We compare the scalability, power efficiency and energy consumption of distributed and shared memory versions of each applications and analyze performance and bottlenecks of different combinations of MPI tasks/OpenMP threads. Our results show that, although shared memory programming models usually provide lower synchronization cost, achieving the highest performance/efficiency requires a combination for MPI tasks/OpenMP threads that is dependent on the underlying architecture and takes into consideration how hardware resources are distributed among the computing elements. More importantly, our results show that the \"best configuration\" strongly depends on the particular target metric.","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114921048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Hackenberg, T. Ilsche, Joseph Schuchart, R. Schöne, W. Nagel, Marc Simon, Yiannis Georgiou
{"title":"HDEEM: High Definition Energy Efficiency Monitoring","authors":"D. Hackenberg, T. Ilsche, Joseph Schuchart, R. Schöne, W. Nagel, Marc Simon, Yiannis Georgiou","doi":"10.1109/E2SC.2014.13","DOIUrl":"https://doi.org/10.1109/E2SC.2014.13","url":null,"abstract":"Accurate and fine-grained power measurements of computing systems are essential for energy-aware performance optimizations of HPC systems and applications. Although cluster wide instrumentation options are available, fine spatial granularity and temporal resolution are not supported by the system vendors and extra hardware is needed to capture the power consumption information. We introduce the High Definition Energy Efficiency Monitoring (HDEEM) infrastructure, a sophisticated approach towards systemwide and fine-grained power measurements that enable energy-aware performance optimizations of parallel codes. Our approach is targeted at instrumenting multiple HPC racks with power sensors that have a sampling rate of about 8 kSa/s as well as finer spatial granularity, e.g., for per-CPU measurements. We specifically focus on the correctness of power measurement samples and energy consumption calculations based on these power samples. We also discuss scalable and low-overhead or overhead-free options for online and offline (post-mortem) processing of power measurement data.","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116248772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anilkumar Nandamuri, A. Malik, A. Qawasmeh, B. Chapman
{"title":"Power and Energy Footprint of OpenMP Programs Using OpenMP Runtime API","authors":"Anilkumar Nandamuri, A. Malik, A. Qawasmeh, B. Chapman","doi":"10.1109/E2SC.2014.11","DOIUrl":"https://doi.org/10.1109/E2SC.2014.11","url":null,"abstract":"Power and energy have become dominant aspects of hardware and software design in the High Performance Computing (HPC). Recently, the Department of Defense (DOD) has put a constraint that applications and architectures need to attain 75 GFLOPS/Watt in order to support the future missions. This requires a significant research effort towards power and energy optimization. OpenMP programming model is an integral part of HPC. Comprehensive analysis of OpenMP programs for power and execution performance is an active research area. Work has been done to characterize OpenMP programs with respect to power performance at kernel level. However, no work has been done at the OpenMP event level. OpenMP Runtime API (ORA), proposed by the OpenMP standard committee, allow a performance tool to collect information at the OpenMP event level. In this paper, we present a comprehensive analysis of the OpenMP programs using ORA for power and execution performance. Using hardware counters in the Intel SandyBridge x86-64 and Running Average Power Limit (RAPL) energy sensors, we measure power and energy characteristics of OpenMP benchmarks. Our results show that the best execution performance does not always give the best energy usage. We also find out that the waiting time at the barriers and in queue are the main factors for high power consumption for a given OpenMP program. Our results also show that there are unique patterns at the fine level that can be used by the dynamic power management system to enhance the power performance. Our results show substantial variation in energy usage depending upon the runtime environment.","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115245129","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Bodas, Justin J. Song, Muralidhar Rajappa, Andy Hoffman
{"title":"Simple Power-Aware Scheduler to Limit Power Consumption by HPC System within a Budget","authors":"D. Bodas, Justin J. Song, Muralidhar Rajappa, Andy Hoffman","doi":"10.1109/E2SC.2014.8","DOIUrl":"https://doi.org/10.1109/E2SC.2014.8","url":null,"abstract":"Future Exascale systems are projected to require tens of megawatts. While facilities must provision sufficient power to realize peak performance, limited power availability will require power capping. Current approaches for power capping limit CPU power state and are agnostic to workload characteristics. Injudicious use of such mechanisms in HPC system can impose a devastating impact on performance. We propose integrating power limiting into a job scheduler. We will describe a power-aware scheduler that monitors power consumption, distributes the power budget to each job, and implements a \"uniform frequency\" mechanism to limit power. We will compare three implementations of uniform frequency. We will show that power monitoring improves the probability of launching a job earlier, allows a job to run faster, and reduces stranded power. Our data shows that \"auto mode\" for uniform frequency operates at 40% higher frequency than a fixed frequency mode.","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128745111","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jacob Combs, Jolie Nazor, Rachelle Thysell, Fabian Santiago, Matthew Hardwick, Lowell Olson, Suzanne Rivoire, C. Hsu, S. Poole
{"title":"Power Signatures of High-Performance Computing Workloads","authors":"Jacob Combs, Jolie Nazor, Rachelle Thysell, Fabian Santiago, Matthew Hardwick, Lowell Olson, Suzanne Rivoire, C. Hsu, S. Poole","doi":"10.1109/E2SC.2014.9","DOIUrl":"https://doi.org/10.1109/E2SC.2014.9","url":null,"abstract":"Workload-aware power management and scheduling techniques have the potential to save energy while minimizing negative impact on performance. The effectiveness of these techniques depends on the stability of a workload's power consumption pattern across different input data, resource allocations (e.g. number of cores), and hardware platforms. In this paper, we show that the power consumption behavior of HPC workloads can be accurately captured by concise signatures built from their power traces. We validate this approach using 255 traces collected from 13 high-performance computing workloads on 4 different hardware platforms. First, we use both feature-based and time-series-based distance metrics to cluster our traces, and we quantitatively show that feature-based clusterings segregate traces by workload just as effectively as the more compute- and space-intensive time-series-based clusterings. Second, we demonstrate that unlabeled traces can be classified by workload with over 85% accuracy, based only on these concise statistical signatures.","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114532051","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Measurement and Interpretation of Micro-benchmark and Application Energy Use on the Cray XC30","authors":"Brian Austin, N. Wright","doi":"10.1109/E2SC.2014.7","DOIUrl":"https://doi.org/10.1109/E2SC.2014.7","url":null,"abstract":"Understanding patterns of application energy use is key to reaching future HPC efficiency goals. We have measured the sensitivity of energy use to CPU frequency for several microbenchmarks and applications on a Cray XC30. We suggest first order models for performance and power vs. frequency and show that these are sufficient to accurately fit the measured energy data. Examination of the resulting energy model shows that an application's energy/frequency profiles have minima only if a) the frequency change crosses an architectural balance point that is performance-critical for the particular application or b) a significant fraction of the runtime is spent in off-chip operations or c) there is sufficient static power drawn to motivate a race-to-halt. All three forms of energy minima are represented in our sample of HPC applications. The energy-optimal frequencies on this architecture are: MILC (1.8 GHz), GTC (3.6 GHz) and Mini-DFT (1.6-1.8 GHz).","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129956276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On the Feasibility of Dynamic Power Steering","authors":"K. Barker, D. Kerbyson, E. Anger","doi":"10.1109/E2SC.2014.6","DOIUrl":"https://doi.org/10.1109/E2SC.2014.6","url":null,"abstract":"While high performance has always been the primary constraint behind large-scale system design, future systems will be built with increasing energy efficiency in mind. Mechanisms such as fine-grained power scaling and gating will provide tools to system-software and application developers to ensure the most efficient use of tightly constrained power budgets. Such approaches to-date have been focused on node-level optimizations to impact overall system energy efficiency. In this work we introduce Dynamic Power Steering, in which power can be dynamically routed across a system to resources where it will be of most benefit and away from other resources to maintain a near-constant overall power budget. This, a higher-level algorithmic approach to improving energy efficiency, considers the whole extent of a system being used by an application. It can be used for applications in which there is load-imbalance that varies over its execution. Using two classes of applications, namely those that contain a wavefront type processing, and a particle-in-cell, we quantify the benefit of Dynamic Power Steering for a variety of workload characteristics and derive some insight into the ways in which workload behavior affect Power Steering applicability.","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"89 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114484668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Energy-Efficient Stencil Computations on Distributed GPUs Using Dynamic Parallelism and GPU-Controlled Communication","authors":"Lena Oden, Benjamin Klenk, H. Fröning","doi":"10.1109/E2SC.2014.14","DOIUrl":"https://doi.org/10.1109/E2SC.2014.14","url":null,"abstract":"GPUs are widely used in high performance computing, due to their high computational power and high performance per Watt. Still, one of the main bottlenecks of GPU-accelerated cluster computing is the data transfer between distributed GPUs. This not only affects performance, but also power consumption. The most common way to utilize a GPU cluster is a hybrid model, in which the GPU is used to accelerate the computation while the CPU is responsible for the communication. This approach always requires a dedicated CPU thread, which consumes additional CPU cycles and therefore increases the power consumption of the complete application. In recent work we have shown that the GPU is able to control the communication independently of the CPU. Still, there are several problems with GPU-controlled communication. The main problem is intra-GPU synchronization, since GPU blocks are non-preemptive. Therefore, the use of communication requests within a GPU can easily result in a deadlock. In this work we show how Dynamic Parallelism solves this problem. GPU-controlled communication in combination with Dynamic Parallelism allows keeping the control flow of multi-GPU applications on the GPU and bypassing the CPU completely. Although the performance of applications using GPU-controlled communication is still slightly worse than the performance of hybrid applications, we will show that performance per Watt increases by up to 10% while still using commodity hardware.","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"106 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124085782","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Trapped Capacity: Scheduling under a Power Cap to Maximize Machine-Room Throughput","authors":"Ziming Zhang, M. Lang, S. Pakin, Song Fu","doi":"10.1109/E2SC.2014.10","DOIUrl":"https://doi.org/10.1109/E2SC.2014.10","url":null,"abstract":"Power-aware parallel job scheduling has been recognized as a demanding issue in the high-performance computing (HPC) community. The goal is to efficiently allocate and utilize power and energy in machine rooms. In practice the power for machine rooms is well over-provisioned, specified by high energy LINPACK runs or nameplate power estimates. This results in a considerable amount of trapped power capacity. Instead of being wasted, this trapped power capacity should be reclaimed to accommodate more compute nodes in the machine room and thereby increase system throughput. But to do this we need the ability to enforce a system-wide power cap. In this paper, we present TracSim, a full-system simulator that enables users to evaluate the performance of different policies for scheduling parallel tasks under a power cap. TracSim simulates the executing environment of an HPC cluster at Los Alamos National Laboratory (LANL). We use real measurements from the LANL cluster to set the configuration parameters of TracSim. TracSim enables users to specify the system topology, hardware configuration, power cap, and task workload, and to develop resource configuration and task scheduling policies aiming to maximize machine-room throughput while keeping power consumption under a power cap by exploiting CPU throttling techniques. We leverage TracSim to implement and evaluate three resource scheduling policies. Simulation results show the performance of those policies and quantify the amount of trapped capacity that can effectively be reclaimed.","PeriodicalId":279092,"journal":{"name":"2014 Energy Efficient Supercomputing Workshop","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126912249","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}