{"title":"Improving generalization performance in character recognition","authors":"H. Drucker, Y. Le Cun","doi":"10.1109/NNSP.1991.239522","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239522","url":null,"abstract":"One test of a new training algorithm is how well the algorithm generalizes from the training data to the test data. A new neural net training algorithm termed double backpropagation improves generalization in character recognition by minimizing the change in the output due to small changes in the input. This is accomplished by minimizing the normal energy term found in backpropagation and an additional energy term that is a function of the Jacobian.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133310143","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Neural networks for extracting unsymmetric principal components","authors":"S. Kung, K. Diamantaras","doi":"10.1109/NNSP.1991.239536","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239536","url":null,"abstract":"The authors introduce two forms of unsymmetric principal component analysis (UPCA), namely the cross-correlation UPCA and the linear approximation UPCA problem. Both are concerned with the SVD of the input-teacher cross-correlation matrix itself (first problem) or after prewhitening (second problem). The second problem is also equivalent to reduced-rank Wiener filtering. For the former problem, the authors propose an unsymmetric linear model for extracting one or more components using lateral inhibition connections in the hidden layer. The numerical convergence properties of the model are theoretically established. For the linear approximation UPCA problem, one can apply back-propagation extended either using a straightforward deflation procedure or with the use of lateral orthogonalizing connections in the hidden layer. All proposed models were tested and the simulation results confirm the theoretical expectations.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115459910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Nonlinear prediction of speech signals using memory neuron networks","authors":"P. Poddar, K. Unnikrishnan","doi":"10.1109/NNSP.1991.239502","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239502","url":null,"abstract":"The authors present a feed-forward neural network architecture that can be used for nonlinear autoregressive prediction of multivariate time-series. It uses specialized neurons (called memory neurons) to store past activations of the network in an efficient fashion. The network learns to be a nonlinear predictor of the appropriate order to model temporal waveforms of speech signals. Arrays of such networks can be used to build real-time classifiers of speech sounds. Experiments where memory-neuron networks are trained to predict speech waveforms and sequences of spectral frames are described. Performance of the network for prediction of time-series with minimal a priori assumptions of its statistical properties is shown to be better than linear autoregressive models.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115999808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Dimensionality reduction of dynamical patterns using a neural network","authors":"S. Nakagawa, Y. Ono, Y. Hirata","doi":"10.1109/NNSP.1991.239516","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239516","url":null,"abstract":"To recognize speech with dynamical features, one should use feature parameters including dynamical changing patterns, that is, time sequential patterns. The K-L expansion has been used to reduce the dimensionality of time sequential patterns. This method changes the axes of feature parameter space linearly by minimizing the error between original and reconstructed parameters. In this paper, the dimensionality of dynamical features is reduced by using one nonlinear dimensional compressing ability of the neural network. The authors compared the proposed method on speech recognition using a continuous HMM (hidden Markov model) with the reduction method using one K-L expansion and the feature parameters of regression coefficients in addition to original static features.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"86 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126452792","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Nonlinear resampling transformation for automatic speech recognition","authors":"Y.D. Liu, Y. Lee, H. Chen, G. Sun","doi":"10.1109/NNSP.1991.239510","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239510","url":null,"abstract":"A new technique for speech signal processing called nonlinear resampling transformation (NRT) is proposed. The representation of a speech pattern derived from this technique has two important features: first, it reduces redundancy; second, it effectively removes the nonlinear variations of speech signals in time. The authors have applied NRT to the TI isolated-word database achieving a 99.66% recognition rate on a 10 digits multi-speaker task for a linear predictive neural net classifier. In their experiment, the authors have also found that discriminative training is superior to nondiscriminative training for linear predictive neural network classifiers.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133946515","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Configuring stack filters by the LMS algorithm","authors":"N. Ansari, Y. Huang, J. Lin","doi":"10.1109/NNSP.1991.239484","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239484","url":null,"abstract":"Stack filters are a class of sliding-window nonlinear digital filters that possess the weak superposition property (threshold decomposition) and the ordering property known as the stacking property. They have been demonstrated to be robust in suppressing noise. A new method based on the least means squares (LMS) algorithm is developed to adaptively configure a stack filter. Experimental results are presented to demonstrate the effectiveness of the proposed method to noise suppression.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130855788","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A neural architecture for nonlinear adaptive filtering of time series","authors":"Nils Hoffmann, J. Larsen","doi":"10.1109/NNSP.1991.239488","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239488","url":null,"abstract":"A neural architecture for adaptive filtering which incorporates a modularization principle is proposed. It facilitates a sparse parameterization, i.e. fewer parameters have to be estimated in a supervised training procedure. The main idea is to use a preprocessor which determines the dimension of the input space and can be designed independently of the subsequent nonlinearity. Two suggestions for the preprocessor are presented: the derivative preprocessor and the principal component analysis. A novel implementation of fixed Volterra nonlinearities is given. It forces the boundedness of the polynominals by scaling and limiting the inputs signals. The nonlinearity is constructed from Chebychev polynominals. The authors apply a second-order algorithm for updating the weights for adaptive nonlinearities. Finally the simulations indicate that the two kinds of preprocessing tend to complement each other while there is no obvious difference between the performance of the ANL and FNL.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126736722","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A multilayer perceptron feature extractor for reading sequenced DNA autoradiograms","authors":"M. Murdock, N. Cotter, R. Gesteland","doi":"10.1109/NNSP.1991.239485","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239485","url":null,"abstract":"The authors report on the application of the three-layer, backward error propagation neural network to the problem of reading sequenced DNA autoradiograms. The network is used for band identification by extracting two features: band intensity level and band intensity gradient. A training set of 16000 12*12 gray scale patterns is generated. Trained with these patterns, the network successfully learned to identify the degree of presence and absence of these two low level features.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"245 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123111065","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Nonlinear adaptive filtering of systems with hysteresis by quantized mean field annealing","authors":"R. A. Nobakht, S. Ardalan, D. van den Bout","doi":"10.1109/NNSP.1991.239526","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239526","url":null,"abstract":"A technique for nonlinear adaptive filtering of systems with hysteresis has been developed which combines quantized mean field annealing (QMFA) and conventional RLS/FTF adaptive filtering. Hysteresis is modeled as a nonlinear system with memory. Unlike other methods which rely on Volterra and Wiener models, this technique can efficiently handle large order nonlinearities with or without hysteresis effects. The nonlinear channel is divided into a memory nonlinearity followed by a dispersive linear system. Assuming that the dispersive linear system is stationary during initialization, and the nonlinearity does not change while the dispersive linear system varies with time, QMFA is applied to obtain the coefficients and the order of the memory of the nonlinearity and RLS/FTF is applied to determine the weights of the dispersive linear system. Application of this method to a full duplex digital subscriber loop is made. Simulations show the superior performance of our technique compared to that of ordinary RLS/FTF and steepest-descent algorithms.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133789896","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A parallel learning filter system that learns the KL-expansion from examples","authors":"R. Lenz, M. Osterberg","doi":"10.1109/NNSP.1991.239529","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239529","url":null,"abstract":"A new method for learning in a single-layer linear neural network is investigated. It is based on an optimality criterion that maximizes the information in the outputs and simultaneously concentrates the outputs. The system consists of a number of so-called basic units and it is shown that the stable states of these basic units correspond to the (pure) eigenvectors of the input correlation matrix. The authors show that the basic units learn in parallel and that the communication between the units is kept to a minimum. They discuss two different implementations of the learning rule, a heuristic one and one based on the Newton-rule. They demonstrate the properties of the system with the help of two classes of examples: waveform analysis and simple OCR-reading. In the waveform-analysis case the eigenfunctions of the systems are known from the group-theoretical studies and the authors show that the system indeed stabilizes in these states.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"173 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133039759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}