Generative Network Correction to Promote Incremental Learning

Abstract

Neural networks are often designed for closed environments that are not open to acquisition of new knowledge. Incremental learning techniques allow neural networks to adapt to changing environments, but these methods often encounter challenges causing models to suffer from low classification accuracies. The main problem faced is catastrophic forgetting and this problem is more harmful when using incremental strategies compared to regular tasks. Some known causes of catastrophic forgetting are weight drift and inter-class confusion; these problems cause the network to erroneously fuse trained classes or to forget a learned class. This paper addresses these issues by focusing on data pre-processing and using network feedback corrections for incremental learning. Data pre-processing is important as the quality of the training data used affects the network's ability to maintain continuous class discrimination. This approach uses a generative model to modify the data input for the incremental model. Network feedback corrections would allow the network to adapt to newly found classes and scale based on network need. With combination of generative data pre-processing and network feedback, this paper proposes an approach for efficient long-term incremental learning. The results obtained are compared with similar state-of-the-art algorithms and show high incremental accuracy levels.