Optimising hyperparameter search in a visual thalamocortical pathway model

Abstract

We have made a comparative study of three optimisation algorithms viz. Random Search (RS), Grid Search (GS) and Bayesian Optimization (BO) to find optimal hyperparameter combinations in an existing brain-inspired thalamocortical model that can simulate brain signals such as local field potentials (lfp) and electroencephalogram (eeg). The layout and parameters for the model are sourced from anatomical and physiological data. However, there is a lot of missing data in such sources due to obvious constraints in wet-lab experimental studies. In our previous work, the missing data are set by trial and error. As the scale of the model gets larger though, the combinatorics of the hyperparameters explode and manual parameter tuning gets non-trivial. The goal of this study is to identify the optimisation algorithm (among the three abovementioned) that gives the best performance at minimal computational costs; performance is evaluated by setting an objective, which is to search for hyperparameter combinations that can simulate theta (4 – 8 Hz), alpha (8 – 13 Hz) and beta (13 – 30 Hz) rhythms, which are typically observed in eeg and lfp. Each optimisation algorithm is tested on a small model (thalamus only) with eight hyperparameters and a large model (thalamocortical) with maximum of fifteen hyperparameters. The performance metric for each algorithm is measured by the number of times the objective is achieved during a fixed number of trials. Our results demonstrate that BO performs the best in reaching the objective with a 30.5% better performance compared to GS and 13% better than RS. In comparison, GS performance is lower with an exponential increase in time with increasing grid size. Overall, our study demonstrates the suitability of using the BO for optimising hyperparameter search in our thalamocortical network model of the visual pathway.