Model-free Optimization: The Exploration-Exploitation Paradigm

Abstract

The trade-off between exploration and exploitation has been crucial in the field of optimization where the cost function is expensive and takes time to converge. The use of Exploration-Exploitation concept in the acquisition function helps to find the optimal values of the objective function completely model-free. Steady-state input-output map of a dynamical system gives an input-output relation that can be utilised to replace the optimizer of the objective function with the control signal. Hence, solving an expensive optimization problem of a control application gradient-free and without injecting perturbation signal. Initial sample size, sampling technique, and the type of acquisition function influences the rate of convergence of the objective function to its optimum. As the sample size increases the function value converges to its optimum faster with less computation time. The use of Expected Improvement as an acquisition function converges the function value closer to its optimum value and gives better-approximated results as compared to other acquisition functions. Model-free optimization using exploration and exploitation can be used widely in data-driven based control application to compute black-box functions solely based on input and output measurements making it computationally less burden.