Novelty detector for reinforcement learning based on forecasting

Abstract

The paper proposes a novelty detector based on an artificial neural network forecaster. It shows how such forecaster can be constructed and as a novelty detector. Two variations of the forecaster are presented - one is based on backpropagation, and the other on Rprop. It is shown how the detector can be used to approach the exploration vs. exploitation trade-off. Experimental results are presented for both versions of the detector along with a comparison with novelty detectors based on the concept of the habituated self-organising map (HSOM). It is shown that learning based on the proposed detector can outperform that using the HSOM-based detector. Finally, the paper identifies several lines along which future research may progress.