Less is More: Dimensionality Analysis of Pure Random Orthogonal Search Through the Lens of Degrees of Freedom

Nature-inspired metaheuristic algorithm plays an autocratic role in optimization (OP). The dominance of metaheuristic algorithms has managed to solicit the focus upon themselves and has overshadowed other types of OP algorithms. Random optimization (RO) is one such type of underrepresented OP algorithm, which commanded significant interest in the mid-’60 s but eventually lost its glitter due to its lackluster OP performance. Pure random orthogonal search (PROS) is a recently published RO algorithm that has revived interest in RO. PROS is a simple, hyperparameter-free OP algorithm capable of dissipating performance better than some established metaheuristic algorithms. Unlike pure random search (PRS), where the optimizer is free to move anywhere within the feasible region, PROS effectively restricts the explorable feasible region to the region strictly orthogonal to the current location, and this restriction immensely boosts its OP performance. Between the two extremes of PRS and PROS, a spectrum of possible movement patterns merits our attention. In this paper, we perform several numerical experiments to study how the freedom to move in different dimensions (Degrees of Freedom) influences the performance of the PRS & PROS algorithm. Further, the notion of an ‘Active Feasible Region’ is introduced to analyze PROS and other related RO algorithms. We propose two simple modifications to the PROS algorithm based on the experiments. The modifications yield marginal performance gains over PROS. Nevertheless, valuable insights are revealed upon the effect of different degrees of freedom and orthogonality constraint and how they could be leveraged to our advantage. The python code is publicly available at: https://github.com/Shahul-Rahman/Less-is-more.