PMCNS: Using a Progressively Stricter Fitness Criterion to Guide Novelty Search

Abstract

Novelty search is an evolutionary approach in which the population is driven towards behavioural innovation instead of towards a fixed objective. The use of behavioural novelty to score candidate solutions precludes convergence to local optima. However, in novelty search, significant effort may be spent on exploration of novel, but unfit behaviours. We propose progressive minimal criteria novelty search (PMCNS) to overcome this issue. In PMCNS, novelty search can freely explore the behaviour space as long as the solutions meet a progressively stricter fitness criterion. We evaluate the performance of our approach by evolving neurocontrollers for swarms of robots in two distinct tasks. Our results show that PMCNS outperforms fitness-based evolution and pure novelty search, and that PMCNS is superior to linear scalarisation of novelty and fitness scores. An analysis of behaviour space exploration shows that the benefits of novelty search are conserved in PMCNS despite the evolutionary pressure towards progressively fitter behaviours.