LIBNVCD: An Extendable and User-friendly Multi-GPU Performance Measurement Tool

Abstract

Cost and power efficiency considerations have driven High Performance Computing (HPC) system design inno-vations in accelerator-based heterogeneous computing. Complex interactions between applications and heterogeneous hardware make it difficult for users to extract maximum performance out of these systems. While there is a plethora of performance measurement and analysis tools for CPU s, the same is not the case for GPUs. Existing tools either provide too high-level information or are overly complicated to setup, impeding performance profiling. While NVIDIA's CUPTI profiling library enables basic kernel-level measurements on NVIDIA's GPUs, it does not provide root-causes of performance slowdown. This paper presents a low-overhead, flexible, and user-friendly tool, LIBNV CD, built on top of CUPTI to simplify performance measurement and analysis of NVIDIA GPUs. LIBNVCD simplifies obtaining fine-grained measurements, requiring only three function calls in source, while masking changes and complexities of CUPTI. By automatically discovering performance event groups, LIBNV CD reduces data collection overhead significantly as many events (not all) can be measured at once. This user-friendly multi-GPU performance measurement tool incurs a mean overhead of less than 1% as compared to CUPTI, and has been released publicly.