Mixture of heterogeneous experts applied to time series: a comparative study

Prediction models for time series generally include preprocessing followed by the synthesis of an input-output mapping. Neural network models have been adopted to perform both steps, by means of unsupervised and supervised learning, respectively. The flexibility and the generalization capability are the most relevant attributes in favor of connectionist approaches. However, even though time series prediction can be roughly interpreted as learning from data, high levels of performance will solely be achieved if some peculiarities of each time series are properly considered in the design, particularly the existence of trend and seasonality. Instead of directly adopting detrend and/or deseasonality treatments, this paper proposes a novel paradigm for supervised learning based on a mixture of heterogeneous experts. Some mixture models have already been proved to produce good performance as predictors, but the present approach is devoted to a hybrid mixture composed of a set of distinct experts. The purpose is not only to further explore the "divide-and-conquer" principle, but also to compare the performance of mixture of heterogeneous experts with the standard mixture of experts approach, using ten distinct time series. The obtained results indicate that mixture of heterogeneous experts generally requires a more elaborate gating device and performs better in the case of more challenging time series.