CMA-ES enhanced optimization of optical nonlinear activation functions

The development of all-optical and electro-optical neural networks represents a rapidly growing field of research. In this field, optimization and reconfigurability of optical nonlinear activation functions (ONAF) devices are crucial. In this work, we propose an optimization framework for MZI and MRR-based ONAF devices using the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). By incorporating this approach into the optimization of artificial neural networks (ANNs), the root mean square error (RMSE) of the fitted optimization functions can reach as low as −86.0 dB. We also present a theoretical analysis of electro-optical (E/O) modulation in directional couplers (DC), enabling reconfigurable power coupling coefficients. Additionally, we investigate an alternative MRR configuration where output light is coupled with MZI from the drop port instead of the through port, leading to optimal performance. This setup allows for the implementation of several activation functions including Parametric ReLU, which attains 100 % training accuracy and 99.1 % validation accuracy in the MNIST handwritten digit recognition task. Our findings provide new insights into the design and optimization of optical activation devices for integrated photonic neural networks.