Minimax optimality of kernel ridge regression when kernel eigenvalues decay polynomially or exponentially

Abstract

We investigate the minimax optimality of the kernel ridge regression by quantifying the estimation complexity owing to the dimensionality under the polynomial or exponential decay rates of the kernel function’s eigenvalues. Based on this result, we elucidate why certain $d$-dimensional spaces allow us to bypass the curse of dimensionality in nonparametric function estimation, because the convergence rates are bounded by those of the univariate case, with a logarithmic factor raised to a power determined by the dimension. Our results reveal that convergence rates with logarithmic factors are generally uniformly unimprovable.