Deep Reinforcement Learning-Based Contract Incentive and Computation Offloading for Mobile Edge Computing-Enabled Blockchain

Abstract

The resolution of proof-of-work problem in blockchain requires significant amount of resources, while the lack of computing power on mobile devices limits the development of blockchain in mobile applications. To mitigate this issue, the combination of blockchain and mobile edge computing (MEC) has attracted much attention. In this paper, we consider an edge-enabled blockchain system that includes one single edge service provider (ESP), multiple types of miners and edge nodes. Each miner submits offloading request to ESP. In response, ESP designs contract to incentivize various types of edge nodes to contribute resources and offer computational services to the miners. This problem is a joint optimization problem of offloading decisions and contract design. Due to the time-variability of network environment, the randomness of miners’ task demands, and the asymmetric information between the ESP and edge nodes, solving this problem is challenging. We propose a deep reinforcement learning contract mechanism (DRLCM) for incentive-based computation offloading strategies, which divides the original problem into two sub-problems: computation offloading and contract design. Initially, the deep Q-network (DQN) algorithm is used to update the offloading decisions based on the evolving task demands and network conditions. Secondly, contract is designed to motivate edge nodes to participate in resource sharing. The problem is simplified by analyzing the necessary and sufficient conditions of feasible contract, and the Lagrange multiplier method is used to approximate the optimal contract. Simulation experiments demonstrate the effectiveness of the DRLCM algorithm, which shows better convergence and performance compared to traditional DQN, Double-DQN algorithms and Dueling-DQN.