Promoting second-harmonic generation in the LiNbO3 film combined with metasurface using plasmonic quasi bound states in the continuum

Metasurfaces have become a fascinating framework for nonlinear optics, with the advantages of a compact footprint and unprecedented flexibility to manipulate light. However, further advancements are necessary to enhance the efficiency of metasurfaces in nonlinear devices. Here, a novel approach for second harmonic generation (SHG) based on the LiNbO₃ metasurface using leaked plasmonic bound states in the continuum (BIC) is proposed. The behavior of SHG in the guided mode resonance (GMR) under TE polarization and plasmonic modes under TM polarization is investigated. The structure consists of a plasmonic grating overlying a nonlinear lithium niobate dielectric waveguide layer that supports two different BIC, namely plasmonic BIC and GMR BIC. The evolution of second harmonics generation(SHG) near two groups of BIC is explored. The SHG of the plasmonic quasi-BIC is stronger than that based on the GMR quasi-BIC. In addition, the plasmonic accidental quasi-BIC produces stronger harmonic effects than the quasi-BIC based on symmetry-broken. Specifically, at a pump intensity of 30 MW/cm², this accidental quasi-BIC results in SHG efficiency of 1.86 × 10⁻³. This work provides a valuable approach to achieving enhanced SHG using plasmonic and BIC. It opens up new possibilities for the utilization of LiNbO₃ in integrated nonlinear nanophotonics and paves the way for the development of advanced nonlinear photonic devices.