Powering Practical Performance: Accelerated Numerical Computing in Pure Python

Abstract

In this paper, we tackle a generic n-dimensional numerical computing problem to compare performance and analyze tradeoffs between popular frameworks using open source Jupyter notebook examples. Most data science practitioners perform their work in Python because of its high-level abstraction and rich set of numerical computing libraries. However, the choice of library and methodology is driven by complexity-impacting constraints like problem size, latency, memory, physical size, weight, power, hardware, and others. To that end, we demonstrate that a wide selection of GPU-accelerated libraries (RAPIDS, CuPy, Numba, Dask), including the development of hand-tuned CUDA kernels, are accessible to data scientists without ever leaving Python. We address the Python developer community by showing C/C++ is not necessary to access single/multi-GPU acceleration for data science applications. We solve a common numerical computing problem - finding the closest point in array B from every point (and its index) in array A, requiring up to 8.8 trillion distance comparisons - on a GPU-equipped workstation without writing a line of C/C++.