Metamaterial is an artificial periodic structure with properties unattainable in natural materials. Since natural materials rarely interact with terahertz (THz) waves, metamaterial has opened up a new way to interact with THz waves and provide a platform for filling the THz frequency gap [1], [2], [3]. The metamaterial devices have attracted tremendous attention as they are designed with active materials, such as graphene [4], [5], [6], [7], [8], [9], [10], [11], vanadium dioxide [12], [13], MoS2 [14], [15], [16], liquid crystals [17], [18], black phosphorus [19], [20], [21], WS2 [22], and Dirac semimetal [23], [24], [25], [26]. Among the active materials, graphene has attracted remarkable attention due to its actively tunable optical properties. Graphene is a monolayer of hexagonally arranged carbon atoms with high electron mobility, excellent thermal and electrical conductivity [27], [28]. The electrical conductivity of graphene can be controlled by chemical doping or the voltage [29]. More importantly, graphene can support surface plasmon resonance in the THz region [30], and this characteristic can enhance the interaction between THz waves and graphene. In addition, the THz metamaterial devices are sensitive to the change in the chemical potential of graphene. Thus, the performance of THz metamaterial devices can be effectively controlled by adjusting the chemical potential, which provides a theoretical basis for designing tunable metamaterial devices [31], [32], [33], [34].

Bound state in the continuum (BIC) originated from quantum mechanics [35], and it has gained enormous attraction due to its effectiveness in exciting resonant peaks with ultra-high quality (Q) factors in recent years. Authentically optical BIC is non-radiating eigenstates embedded in the radiation continuum with an infinite Q factor and zero linewidth [36], [37]. BIC generally includes symmetry-protected BIC [38], [39] and accidental BIC [40], [41]. To employ BIC in real optical systems, a so-called quasi-BIC is transformed from symmetry-protected BIC by breaking symmetry, which can obtain extremely high Q value and ultra-narrow resonant linewidth [42], [43]. Inspired by this physical mechanism, various quasi-BIC-induced metamaterials with high Q factors have been investigated recently. For example, Romano et al. designed an all-dielectric metasurface supporting BIC for sensing in the visible light region [44]. Bai et al. designed that the creation of a vortex beam can be achieved by using the BIC supported by a photonic crystal slab structure [45]. Li et al. studied an all-dielectric metasurface of the double resonances of toroidal dipole and magnetic dipole quasi-BIC for sensing application [46]. Shen et al. demonstrated an all-metal metasurface with quasi-BIC resonance, which can selectively and near-perfectly absorb one circularly polarized THz wave [47]. Yin et al. designed a metal–insulator–metal absorber supported by the quasi-BIC for sensing in the THz region [48]. All-dielectric or all-metal metasurface-based structures typically have the same shortcoming once after the fabrication, the metasurface devices can only work under a single status. To overcome this shortage, recent researchers have tried to study metasurface devices with graphene to achieve dynamic control devices [49], [50]. Li et al. designed a type of graphene-metal hybrid metasurface for optical modulation [51]. Sang et al. showed a graphene photonic crystal slab for highly efficient light absorption and sensing application using Quasi-BIC resonance [52]. Wang et al. used a structure with diagonal round holes to study the properties of BIC excitation and tunable line width of absorption in the infrared spectral region. By doping graphene, it is possible to control the amount of light that is absorbed in the quasi-BIC resonance [53]. To sum up, dynamic controllable quasi-BIC devices are the focus of research. However, the graphene-dielectric hybrid metamaterial (GDHM) with ultra-high Q factors induced by quasi-BIC remains less reported in the THz region.

THz technology has attracted much attention in a large number of applications, including biomedical diagnosis [54], label-free sensing [55], and material analysis [56]. The application of THz technology has already been constrained and hampered by its low sensitivity and Q factor. The BIC can be coupled to the extended state by introducing a perturbation to the resonator structure, leading to the identification of a leakage mode (quasi-BIC) with a high Q factor and a finite narrow bandwidth. In recent years, graphene has become a promising active material for changing the frequency of surface plasmon resonance and getting dynamically tunable resonance peaks [57], [58]. Therefore, it is meaningful to design graphene metamaterial devices by combining quasi-BIC resonance and graphene in the THz region.

With the inspiration of previous studies, we propose a type of GDHM that can support quasi-BIC resonance mode. The quasi-BIC is excited by breaking the symmetry of the GDHM. The analysis of multipoles confirms that the quasi-BIC is dominated by the electric quadrupole mode. The dynamic modulation of the resonance is investigated in combination with graphene. When the losses of LiTaO3 are not considered. The modulation depth can achieve approximately 97% with the chemical potential changing from 0 meV to 16 meV. In addition, the GDHM can function as a refractive index sensor. The GDHM exhibits better optical modulation and sensing performance. When the losses of LiTaO3 are considered. The modulation depth and sensitivity achieve 90% and 309.54 GHz/RIU, respectively, and the maximum FOM is 121.39RIU−1. This work provides a new approach for dynamic optical modulation and sensing application in THz region.