CIRG@UP OptiBench: a statistically sound framework for benchmarking optimisation algorithms

Abstract

This article is a proposal, by the Computational Intelligence Research Group at the University of Pretoria (CIRG@UP), for a framework to benchmark optimisation algorithms. This framework, known as OptiBench, was conceived out of the necessity to consolidate the efforts of a large research group. Many problems arise when different people work independently on their own research initiatives. These problems range from duplicating effort to, more seriously, having conflicting results. In addition, less experienced members of the group are sometimes unfamiliar with the necessary statistical methods required to properly analyse their results. These problems are not limited internally to CIRG@UP but are also prevalent in the research community at large. This proposal aims to standardise the research methodology used by CIRG@UP internally (initially in the optimisation subgroup and later in subgroups working in other paradigms of computational research). Obviously this article cannot dictate the methodologies that should be used by other members of the broader research community, however, the hope is that this framework can be found useful and that others would willingly contribute and become involved.