A Probabilistic Model-Free Approach in Learning Multivariate Noisy Linear Systems

Abstract

The paper provides a series of results concerning the learning from data a linear regressive model in a multivariate framework. The parameter estimates of the regressive model are determined using the maximum likelihood principle and the adaptive learning algorithms are derived using the gradient ascent technique. The predicted output is expressed as the sum of a linear combination of the entries of the input and the random vector that represents the effects of the unobservable factors and noise. In the second section of the paper the mathematical arguments for the estimation scheme based exclusively on a finite size set of observations is provided. The third section of the paper is focused on experimental evaluation of the quality of the resulted learning scheme in order to establish conclusions concerning their accuracy and generalization capacities, the evaluation being performed in terms of metric, probabilistic and informational criterion functions. The final section of the paper contains a series of conclusions and suggestions for further work.