{"title":"Probability estimation by feed-forward networks in continuous speech recognition","authors":"S. Renals, N. Morgan, H. Bourlard","doi":"10.1109/NNSP.1991.239511","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239511","url":null,"abstract":"The authors review the use of feedforward neural networks as estimators of probability densities in hidden Markov modelling. In this paper, they are mostly concerned with radial basis functions (RBF) networks. They not the isomorphism of RBF networks to tied mixture density estimators; additionally they note that RBF networks are trained to estimate posteriors rather than the likelihoods estimated by tied mixture density estimators. They show how the neural network training should be modified to resolve this mismatch. They also discuss problems with discriminative training, particularly the problem of dealing with unlabelled training data and the mismatch between model and data priors.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127250044","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Speech recognition using time-warping neural networks","authors":"K. Aikawa","doi":"10.1109/NNSP.1991.239508","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239508","url":null,"abstract":"The author proposes a time-warping neural network (TWNN) for phoneme-based speech recognition. The TWNN is designed to accept phonemes with arbitrary duration, whereas conventional phoneme recognition networks have a fixed-length input window. The purpose of this network is to cope with not only variability of phoneme duration but also time warping in a phoneme. The proposed network is composed of several time-warping units which each have a time-warping function. The TWNN is characterized by time-warping functions embedded between the input layer and the first hidden layer in the network. The proposed network demonstrates higher phoneme recognition accuracy than a baseline recognizer based on conventional feedforward neural networks and linear time alignment. The recognition accuracy is even higher than that achieved with discrete hidden Markov models.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131719806","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Adaptive neural filters","authors":"L. Yin, J. Astola, Y. Neuvo","doi":"10.1109/NNSP.1991.239491","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239491","url":null,"abstract":"The authors introduce a new class of nonlinear filters called neural filters based on the threshold decomposition and neural networks. Neural filters can approximate both linear FIR filters and weighted order statistic (WOS) filters which include median, rank order, and weighted median filters. An adaptive algorithm is derived for determining optimal neural filters under the mean squared error (MSE) criterion. Experimental results demonstrate that if the input signal is corrupted by Gaussian noise adaptive neural filters converge to linear filters and if corrupted by impulsive noise, optimal neural filters become WOS filters.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116754076","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A neural network pre-processor for multi-tone detection and estimation","authors":"S.S. Rao, S. Sethuraman","doi":"10.1109/NNSP.1991.239483","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239483","url":null,"abstract":"A parallel bank of neural networks each trained in a specific band of the spectrum is proposed as a pre-processor for the detection and estimation of multiple sinusoids at low SNRs. A feedforward neural network model in the autoassociative mode, trained using the backpropagation algorithm, is used to construct this sectionized spectrum analyzer. The key concept behind this scheme is that, the network when trained for a certain spectral band, serves as an excellent filter with sharp transition and near complete attenuation in stopband, even at low SNRs. Simulation results to support the advantages of the proposed scheme are presented. Statistical measurements to determine its degree of reliability in detection have been made.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122725131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Neural-network architecture for linear and nonlinear predictive hidden Markov models: application to speech recognition","authors":"L. Deng, K. Hassanein, M. Elmasry","doi":"10.1109/NNSP.1991.239500","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239500","url":null,"abstract":"A speech recognizer is developed using a layered neural network to implement speech-frame prediction and using a Markov chain to modulate the network's weight parameters. The authors postulate that speech recognition accuracy is closely linked to the capability of the predictive model in representing long-term temporal correlations in data. Analytical expressions are obtained for the correlation functions for various types of predictive models (linear, nonlinear, and jointly linear and nonlinear) in order to determine the faithfulness of the models to the actual speech data. The analytical results, computer simulations, and speech recognition experiments suggest that when nonlinear and linear prediction are jointly performed within the same layer of the neural network, the model is better able to capture long-term data correlations and consequently improve speech recognition performance.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131813256","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The outlier process (picture processing)","authors":"D. Geiger, R.A.M. Pereira","doi":"10.1109/NNSP.1991.239535","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239535","url":null,"abstract":"The authors discuss the problem of detecting outliers from a set of surface data. They start from the Bayes approach and the assumption that surfaces are piecewise smooth and corrupted by a combination of white Gaussian and salt and pepper noise. They show that such surfaces can be modelled by introducing an outlier process that is capable of 'throwing away' data. They make use of mean field techniques to finally obtain a deterministic network. The experimental results with real images support the model.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132229426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Discriminative multi-layer feed-forward networks","authors":"S. Katagiri, C.-H. Lee, B. Juang","doi":"10.1109/NNSP.1991.239540","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239540","url":null,"abstract":"The authors propose a new family of multi-layer, feed-forward network (FFN) architectures. This framework allows examination of several feed-forward networks, including the well-known multi-layer perceptron (MLP) network, the likelihood network (LNET) and the distance network (DNET), in a unified manner. They then introduce a novel formulation which embeds network parameters into a functional form of the classifier design objective so that the network's parameters can be adjusted by gradient search algorithms, such as the generalized probabilistic descent (GPD) method. They evaluate several discriminative three-layer networks by performing a pattern classification task. They demonstrate that the performance of a network can be significantly improved when discriminative formulations are incorporated into the design of the pattern classification networks.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130231854","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A comparison of second-order neural networks to transform-based method for translation- and orientation-invariant object recognition","authors":"R. Duren, B. Peikari","doi":"10.1109/NNSP.1991.239518","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239518","url":null,"abstract":"Neural networks can use second-order neurons to obtain invariance to translations in the input pattern. Alternatively transform methods can be used to obtain translation invariance before classification by a neural network. The authors compare the use of second-order neurons to various translation-invariant transforms. The mapping properties of second-order neurons are compared to those of the general class of fast translation-invariant transforms introduced by Wagh and Kanetkar (1977) and to the power spectra of the Walsh-Hadamard and discrete Fourier transforms. A fast transformation based on the use of higher-order correlations is introduced. Three theorems are proven concerning the ability of various methods to discriminate between similar patterns. Second-order neurons are shown to have several advantages over the transform methods. Experimental results are presented that corroborate the theory.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"2012 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114505110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An effective method for visual pattern recognition","authors":"Ioannis N. M. Papadakis","doi":"10.1109/NNSP.1991.239520","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239520","url":null,"abstract":"An effective method for neural network based visual pattern recognition is presented. It is shown that it can be successfully used for visual recognition of deformed letters. The main advantages of the presented method are its intuitive appeal, simple implementation and analytical justification.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"128 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126063051","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A critical overview of neural network pattern classifiers","authors":"R. Lippmann","doi":"10.1109/NNSP.1991.239515","DOIUrl":"https://doi.org/10.1109/NNSP.1991.239515","url":null,"abstract":"A taxonomy of neural network pattern classifiers is presented which includes four major groupings. Global discriminant classifiers use sigmoid or polynomial computing elements that have 'high' nonzero outputs over most of their input space. Local discriminant classifiers use Gaussian or other localized computing elements that have 'high' nonzero outputs over only a small localized region of their input space. Nearest neighbor classifiers compute the distance to stored exemplar patterns and rule forming classifiers use binary threshold-logic computing elements to produce binary outputs. Results of experiments are presented which demonstrate that neural network classifiers provide error rates which are equivalent to and sometimes lower than those of more conventional Gaussian. Gaussian mixture, and binary three classifiers using the same amount of training data.<<ETX>>","PeriodicalId":354832,"journal":{"name":"Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1991-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127710179","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}