Nearest data processing in GPU

Abstract

Memory wall is known as one of the most critical bottlenecks in processors, rooted in the long memory access delay. With the advent of emerging memory-intensive applications such as image processing, the memory wall problem has become even more critical. Near data processing (NDP) has been introduced as an astonishing solution where instead of moving data from the main memory, instructions are offloaded to the cores integrated with the main memory level. However, in NDP, instructions that are to be offloaded, are statically selected at the compilation time prior to run-time. In addition, NDP ignores the benefit of offloading instructions into the intermediate memory hierarchy levels. We propose Nearest Data Processing (NSDP) which introduces a hierarchical processing approach in GPU. In NSDP, each memory hierarchy level is equipped with processing cores capable of executing instructions. By analyzing the instruction status at run-time, NSDP dynamically decides whether an instruction should be offloaded to the next level of memory hierarchy or be processed at the current level. Depending on the decision, either data is moved upward to the processing core or the instruction is moved downward to the data storage unit. With this approach, the data movement rate has been reduced, on average, by 47 % over the baseline. Consequently, NSDP has been able to improve the system performance, on average, by 37 % and reduce the power consumption, on average, by 18 %.