Moments of traces of random symplectic matrices and hyperelliptic $L$-functions

We study matrix integrals of the form $$\int_{\mathrm{USp(2n)}}\prod_{j=1}^k\mathrm{tr}(U^j)^{a_j}\mathrm d U,$$ where $a_1,\ldots,a_r$ are natural numbers and integration is with respect to the Haar probability measure. We obtain a compact formula (the number of terms depends only on $\sum a_j$ and not on $n,k$) for the above integral in the non-Gaussian range $\sum_{j=1}^kja_j\le 4n+1$. This extends results of Diaconis-Shahshahani and Hughes-Rudnick who obtained a formula for the integral valid in the (Gaussian) range $\sum_{j=1}^kja_j\le n$ and $\sum_{j=1}^kja_j\le 2n+1$ respectively. We derive our formula using the connection between random symplectic matrices and hyperelliptic $L$-functions over finite fields, given by an equidistribution result of Katz and Sarnak, and an evaluation of a certain multiple character sum over the function field $\mathbb F_q(x)$. We apply our formula to study the linear statistics of eigenvalues of random unitary symplectic matrices in a narrow bandwidth sampling regime.