Rademacher learning rates for iterated random functions

Abstract

Most supervised learning methods assume training data is drawn from an i.i.d. sample. However, real-world problems often exhibit temporal dependence and strong correlations between marginals of the data-generating process, rendering the i.i.d. assumption unrealistic. Such cases naturally involve time-series processes and Markov chains. The learning rates typically obtained in these settings remain independent of the data distribution, potentially leading to restrictive hypothesis classes and suboptimal sample complexities. We consider training data generated by an iterated random function that need not be irreducible or aperiodic. Assuming the governing function is contractive in its first argument and subject to certain regularity conditions on the hypothesis class, we first establish uniform convergence for the sample error. We then prove learnability of approximate empirical risk minimization and derive its learning rate bound. Both bounds depend explicitly on the data distribution through the Rademacher complexities of the hypothesis class, thereby better capturing properties of the data-generating distribution.