Simultaneous estimation and variable selection for a non-crossing multiple quantile regression using deep neural networks

Abstract

In this paper, we present the DNN-NMQR estimator, an approach that utilizes a deep neural network structure to solve multiple quantile regression problems. When estimating multiple quantiles, our approach leverages the structural characteristics of DNN to enhance estimation results by encouraging shared learning across different quantiles through DNN-NMQR. Also, this method effectively addresses quantile crossing issues through the penalization method. To refine our methodology, we introduce a convolution-type quadratic smoothing function, ensuring that the objective function remains differentiable throughout. Furthermore, we provide a brief discussion on the convergence analysis of DNN-NMQR, drawing on the concept of the neural tangent kernel. For a high-dimensional case, we propose the (A)GDNN-NMQR estimator, which applies group-wise \(L_1\)-type regularization methods and enjoys the advantages of quantile estimation and variable selection simultaneously. We extensively validate all of our proposed methods through numerical experiments and real data analysis.