A 0.45-mm² 3.49-TOPS/W Cryogenic Deep Reinforcement Learning Module for End-to-End Integrated Circuits Control

Abstract

This work presents a fully unrolled on-chip deep reinforcement learning (DRL) module with a deep Q-network (DQN) and its system integration for integrated circuits control and functionality augmentation tasks, including voltage regulation of a cryogenic single-input triple-output dc–dc converter and recovery of RF fingerprints (RFFs) using a reconfigurable power amplifier (PA) under temperature variations. The complete DRL module features 6-bit fixed-point model parameters, 116 kB of memory, and 128 processing elements. It is equipped with on-chip training capabilities, fully unrolled on a 0.45- ${\mathrm { mm}}^{2}$ core area using 28-nm technology. The design achieves an efficiency of 0.12 nJ per action and a control latency of $4.925~\mu $ s, with a maximum operational efficiency of 3.49 TOPS/W. Temperature effects on the chip are thoroughly demonstrated across a wide temperature range from 358 K ( $85~^{\circ }$ C) to 4.2 K (– $269~^{\circ }$ C).