On the fields of definition of Hodge loci

A polarizable variation of Hodge structure over a smooth complex quasi projective variety $S$ is said to be defined over a number field $L$ if $S$ and the algebraic connection associated to the variation are both defined over $L$. Conjecturally any special subvariety (also called an irreducible component of the Hodge locus) for such variations is defined over $\overline{\mathbb{Q}}$, and its Galois conjugates are also special subvarieties. We prove this conjecture for special subvarieties satisfying a simple monodromy condition. As a corollary we reduce the conjecture that special subvarieties for variation of Hodge structures defined over a number field are defined over $\overline{\mathbb{Q}}$ to the case of special points.