LLM-Based Data Augmentation Method in Reinforcement Learning With Machine-Unlearning and Fine-Tuning

Abstract

Data augmentation in reinforcement learning (RL) aims to generate diverse and extensive datasets to enhance the learning process. Most existing studies on RL augmentation employ sample-based approaches that modify existing samples. However, many of these methods directly adopt augmentation strategies from other domains, which may present limitations when applied to RL. For instance, approaches derived from computer vision techniques are often not well-suited for general RL applications that do not involve visual inputs. Moreover, such sample-based methods frequently overlook the broader characteristics of the training environment, rendering the augmented data potentially less effective in complex scenarios, such as intelligent transportation systems. In addition, these methods can introduce significant risks to critical and sensitive data. For example, introducing noise to samples as a method of augmentation can precipitate adversarial attacks. Thus, a reliable and stable method of augmentation is necessary. To address these concerns, we propose a novel large language model (LLM)-based augmentation strategy in RL with machine unlearning and fine-tuning. This method utilizes LLMs for data augmentation, ensuring the reliability of the augmented data by tailoring it to specific environmental contexts. Additionally, it enhances the quality of augmentation by mitigating the impact on critical samples through the proposed novel machine unlearning method, while simultaneously fine-tuning the model to improve overall performance. Our experimental results indicate that this innovative approach significantly surpasses traditional augmentation methods in learning performance. By mitigating the impact on critical samples, our strategy not only generates more reliable augmented data but also enhances the effectiveness of RL models.