Algorithm comparisons and the significance of population size

Abstract

In studies that compare the performance of population-based optimization algorithms, it is sometimes assumed that the comparison is valid as long as the number of function evaluations is equal, even if the population size differs. This paper shows that such comparisons are invalid. The performance of two algorithms: differential evolution (DE) and global best particle swarm optimization (gbest PSO) are tested on standard benchmark problems with different numbers of individuals/particles (20, 50 and 100). It is shown that there are significance differences in the performance of the same algorithm with the same number of function evaluations, but with different numbers of individuals/particles. Comparisons of different algorithms should therefore always use the same population size for results to be valid.