On measuring the maturity of SYCL implementations by tracking historical performance improvements

SYCL is a platform agnostic, single-source, C++ based, parallel programming framework for developing platform independent software for heterogeneous systems. As an emerging framework, SYCL has been under active development for several years, with multiple implementations available from hardware vendors and others. A crucial metric for potential adopters is how mature these implementations are; are they still improving rapidly, indicating that the space is still quite immature, or has performance improvement plateaued, potentially indicating a mature market? This study presents a historical study of the performance delivered by all major SYCL implementations on a range of supported platforms. We use existing HPC-style mini-apps written in SYCL, and benchmark these on current and historical revisions of each SYCL implementation, revealing the rate of change of performance improvements over time. The data indicates that most SYCL implementations are now quite mature, showing rapid performance improvements in the past, slowing to more modest performance improvements more recently. We also compare the most recent SYCL performance to existing well established frameworks, such as OpenCL and OpenMP.