The rates of convergence of neural network estimates of hierarchical interaction regression models

Abstract

Regression estimation often suffers from the curse of dimensionality. In the present paper we circumvent this problem by introducing a class of models called hierarchical interaction models where the values of a function m : ℝd → ℝ are computed in a feed-forward manner in several layers, where in each layer a function of at most d* inputs produced by the previous layer is computed. We introduce regression estimates based on neural networks with two hidden layers and apply them to estimation of regression functions from a class of hierarchical interaction models. Under smoothness condition imposed on all functions occurring in the model we show that the rate of convergence of these estimates depends on d*, which is typically much smaller than d.