Cascade network architectures

A novel incremental cascade network architecture based on error minimization is presented. The properties of this and related cascade architectures are discussed, and the influence of the objective function is investigated. The performance of the network is achieved by several layers of nonlinear units that are trained in a strictly feedforward manner and one after the other. Nonlinearity is generated by using sigmoid units and, optionally, additional powers of their activity values. Extensive benchmarking results for the XOR problem are reported, as are various classification tasks, and time series prediction. These are compared to other results reported in the literature. Direct cascading is proposed as promising approach to introducing context information in the approximation process.<>