Optimizing Fusion Skeleton for Multimodal Neural Network by Sequential Sampling to Efficiently Estimate Electromagnetic Response of Metasurfaces

Abstract

The skeleton to fuse features from both binarized patterns and vectorized structural parameters of metasurfaces is proposed to be optimized by the sequential sampling method for a better estimation of electromagnetic (EM) responses. By training a surrogate model with prior observations to estimate the posterior performance of fusion skeletons, the proposed method recognizes more promising fusion skeletons and samples them with a higher probability, and the sampled skeletons are used to further update the surrogate for a better estimation. To balance the exploration and exploitation during sampling iterations, a temperature-annealed sampling strategy is adopted. The proposed method is validated by optimizing the fusion skeleton of a multimodal model to estimate the EM responses of a complex frequency-selective rasorber and a metasurface absorber. Numeric results demonstrate that satisfying fusion skeletons can be found within several iterations so that the accuracy and efficiency of estimating EM responses are enhanced. A rough guideline can also be drawn that the fusion between late features from vectorized structural parameters and early features from binarized patterns should be prioritized for better estimation performance.