Accelerating Deep Reinforcement Learning via Phase-Level Parallelism for Robotics Applications

IF 1.4 3区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

IEEE Computer Architecture Letters Pub Date : 2023-12-11 DOI:10.1109/LCA.2023.3341152

Yang-Gon Kim;Yun-Ki Han;Jae-Kang Shin;Jun-Kyum Kim;Lee-Sup Kim

引用次数: 0

Abstract

Deep Reinforcement Learning (DRL) plays a critical role in controlling future intelligent machines like robots and drones. Constantly retrained by newly arriving real-world data, DRL provides optimal autonomous control solutions for adapting to ever-changing environments. However, DRL repeats inference and training that are computationally expensive on resource-constraint mobile/embedded platforms. Even worse, DRL produces a severe hardware underutilization problem due to its unique execution pattern. To overcome the inefficiency of DRL, we propose Train Early Start , a new execution pattern for building the efficient DRL algorithm. Train Early Start parallelizes the inference and training execution, hiding the serialized performance bottleneck and improving the hardware utilization dramatically. Compared to the state-of-the-art mobile SoC, Train Early Start achieves 1.42x speedup and 1.13x energy efficiency.

查看原文本刊更多论文

通过阶段级并行性加速机器人应用中的深度强化学习

深度强化学习（DRL）在控制机器人和无人机等未来智能机器方面发挥着至关重要的作用。DRL 不断根据新到达的真实世界数据进行训练，为适应不断变化的环境提供最佳自主控制解决方案。然而，在资源受限的移动/嵌入式平台上，DRL 需要重复推理和训练，计算成本高昂。更糟糕的是，由于 DRL 独特的执行模式，会产生严重的硬件利用率不足问题。为了克服 DRL 的低效问题，我们提出了一种新的执行模式--Train Early Start，用于构建高效的 DRL 算法。Train Early Start 将推理和训练执行并行化，隐藏了串行化的性能瓶颈，显著提高了硬件利用率。与最先进的移动 SoC 相比，Train Early Start 的速度提高了 1.42 倍，能效提高了 1.13 倍。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Computer Architecture Letters COMPUTER SCIENCE, HARDWARE & ARCHITECTURE-

CiteScore

4.60

自引率

4.30%

发文量

期刊介绍： IEEE Computer Architecture Letters is a rigorously peer-reviewed forum for publishing early, high-impact results in the areas of uni- and multiprocessor computer systems, computer architecture, microarchitecture, workload characterization, performance evaluation and simulation techniques, and power-aware computing. Submissions are welcomed on any topic in computer architecture, especially but not limited to: microprocessor and multiprocessor systems, microarchitecture and ILP processors, workload characterization, performance evaluation and simulation techniques, compiler-hardware and operating system-hardware interactions, interconnect architectures, memory and cache systems, power and thermal issues at the architecture level, I/O architectures and techniques, independent validation of previously published results, analysis of unsuccessful techniques, domain-specific processor architectures (e.g., embedded, graphics, network, etc.), real-time and high-availability architectures, reconfigurable systems.