Deep reinforcement learning for solving the stochastic e-waste collection problem

Abstract

With the growing influence of the internet and information technology, Electrical and Electronic Equipment (EEE) has become a gateway to technological innovations. However, discarded devices, also called e-waste, pose a significant threat to the environment and human health if not properly treated, disposed of, or recycled. In this study, we extend a novel model for the e-waste collection in an urban context: the Heterogeneous VRP with Multiple Time Windows and Stochastic Travel Times (HVRP-MTWSTT). We propose a solution method that employs deep reinforcement learning to guide local search heuristics (DRL-LSH). The contributions of this paper are as follows: (1) HVRP-MTWSTT represents the first stochastic VRP in the context of the e-waste collection problem, incorporating complex constraints such as multiple time windows across a multi-period horizon with a heterogeneous vehicle fleet, (2) The DRL-LSH model uses deep reinforcement learning to provide an online adaptive operator selection layer, selecting the appropriate heuristic based on the search state. The computational experiments demonstrate that DRL-LSH outperforms the state-of-the-art hyperheuristic method by 24.26% on large-scale benchmark instances, with the performance gap increasing as the problem size grows. Additionally, to demonstrate the capability of DRL-LSH in addressing real-world problems, we tested and compared it with reference metaheuristic and hyperheuristic algorithms using a real-world e-waste collection case study in Singapore. The results showed that DRL-LSH significantly outperformed the reference algorithms on a real-world instance in terms of operating profit.