Efficient Estimation of Single-index Models with Deep ReQU Neural Networks

Single-index model offers the greater flexibility of modelling than generalized linear models and also retains the interpretability of the model to some extent. Although many standard approaches such as kernels or penalized/smooothing splines were proposed to estimate smooth link function, they cannot approximate complicated unknown link functions together with the corresponding derivatives effectively due to their poor approximation ability for a finite sample size. To alleviate this problem, this paper proposes a semiparametric least squares estimation approach for a single-index model using the rectifier quadratic unit (ReQU) activated deep neural networks, called deep semiparametric least squares (DSLS) estimation method. Under some regularity conditions, we show non-asymptotic properties of the proposed DSLS estimator, and evidence that the index coefficient estimator can achieve the semiparametric efficiency. In particular, we obtain the consistency and the convergence rate of the proposed DSLS estimator when response variable is conditionally sub-exponential. This is an attempt to incorporate deep learning technique into semiparametrically efficient estimation in a single index model. Several simulation studies and a real example data analysis are conducted to illustrate the proposed DSLS estimator.