Efficient Reinforcement Learning Framework for Automated Logic Synthesis Exploration

Abstract

Logic synthesis is a crucial step in electronic design automation tools for integrated circuit design. In recent years, the development of reinforcement learning (RL) has enabled the designers to automatically explore the logic synthesis process. Existing RL based methods typically use conventional on-policy models, which leads to data inefficiency. Moreover, the exploration approach for FPGA technology mapping in recent works lacks the flexibility of the learning process. In this work, we propose ESE, a reinforcement learning based framework to efficiently learn the logic synthesis process. The framework supports the modeling for both the logic optimization and the FPGA technology mapping. The reward functions and terminal conditions in the RL environment are designed to efficiently guide the optimization of the metrics and execution time. For the modeling of FPGA mapping, the logic optimization and technology mapping are combined to be learned in a flexible way. Moreover, the Proximal Policy Optimization model is adopted to improve the utilization of samples. The proposed framework is evaluated on several common benchmarks. For the logic optimization on the EPFL benchmark, compared with previous works, the proposed method obtains an 11.3% improvement in the average quality (node-level-product) and reduces the execution time by 13.7%. For the FPGA technology mapping on the VTR benchmark, our method improves the average quality (LUT-level-product) by 14.8%, and reduces the execution time by 14.4% compared with the recent work.