Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality

Abstract

This paper considers deep learning from strongly mixing observations and performs a sparse-penalized regularization for deep neural networks (DNN) predictors. In a general framework that includes regression and classification, oracle inequalities for the expected excess risk are established, and upper bounds on the class of Hölder smooth functions and composition structured Hölder functions are provided. For nonparametric autoregression with the Gaussian and Laplace errors, and the Huber loss function, it is shown that the sparse-penalized DNN estimator proposed is optimal (up to a logarithmic factor) in the minimax sense. Based on the lower bound established in Alquier and Kengne (2024), we show that the proposed DNN estimator for the classification task with the logistic loss on strongly mixing observations achieves (up to a logarithmic factor), the minimax optimal convergence rate.