{"title":"On variable sizes and sigmoid activation functions of multilayer perceptrons","authors":"G. Daqi, Liu Hua, Li Changwu","doi":"10.1109/IJCNN.2003.1223717","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1223717","url":null,"abstract":"This paper studies the influences of variable scales and sigmoid activation functions on the performances of multi-layer perceptrons. Generally speaking, it is not certainly suitable to normalize the input data or make the sizes of input variables in the range of [0.0, 1.0]. The viewpoint is explained in details according to the theory of support vector machine (SVM). The convergence and generalization abilities of multilayer perceptrons can be evidently improved by means of the following three methods: (A). Enlarge the sizes of the variable components in the range of [0.0, 3.0]. (B). Change the standard sigmoid activation function f(x)=(1+exp(-x))/sup -1/ into f(x)=3(1+exp(-x/3))/sup -1/. (C). Introduce the sum-of-squares weight term W/sup T/W into the error functions. The classification experiment shows that more than a learning round should be done and the perceptron with the best good generalization performance be held back.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115120602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Prediction of white noise time series using artificial neural networks and asymmetric cost functions","authors":"S. Crone","doi":"10.1109/IJCNN.2003.1223950","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1223950","url":null,"abstract":"Artificial neural networks in time series prediction generally minimise a symmetric statistical error, such as the sum of squared errors, to learn relationships from the presented data. However, applications in business elucidate that real forecasting problems contain non-symmetric errors. The costs arising from suboptimal business decisions based on over-versus underprediction are dissimilar for errors of identical magnitude. To reflect this, a set of asymmetric cost functions is used as objective functions for neural network training, deriving superior forecasts even for white noise time series. Some experimental results are computed using a multilayer perceptron trained with various asymmetric cost functions, evaluating the performance in competition to conventional forecasting methods on a white noise time series extracted from the popular airline passenger data.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116998090","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Bayesian regularized neural network for multiple gene expression pattern classification","authors":"A. Kelemen, Yulan Liang","doi":"10.1109/IJCNN.2003.1223441","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1223441","url":null,"abstract":"We developed Bayesian regularized neural network (BRNN) to characterize multiple gene expression temporal patterns from microarray experiments. One of its attractive property is that it takes into account both the high level noisy feature from microarray data and the uncertainties of the multiple models uniformly in order to avoid overfitting and to improve the generalization performance. Results are encouraging and comparison study with other popular methods is provided.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"110 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117210172","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A hierarchical neuro-fuzzy system based on S-implications","authors":"R. Nowicki, R. Scherer, L. Rutkowski","doi":"10.1109/IJCNN.2003.1223365","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1223365","url":null,"abstract":"In this paper, we present a neuro-fuzzy structure of the hierarchical prioritized structure (HPS) proposed by Yager. The HPS allows for easy hierarchization of a fuzzy rule-base. Our neuro-fuzzy system can be learned by the backpropagation algorithm and is relatively computationally efficient.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"24 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120903730","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Extended simulated annealing for augmented TSP and multi-salesmen TSP","authors":"Chi-Hwa Song, K. Lee, W. Lee","doi":"10.1109/IJCNN.2003.1223777","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1223777","url":null,"abstract":"An extended simulated annealing (ESA), based on grand canonical ensemble (GCE), is proposed. An ESA is used to solve the augmented traveling salesman problems (ATSP) and the multiple traveling salesmen problems. Experimental results show that ESA has salient features such as simplicity and ability to find high-quality solutions as simulated annealing has.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120941116","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The self-organization by lateral inhibition model: validation of clustering","authors":"B. Tang, M. Heywood, M. Shepherd","doi":"10.1109/IJCNN.2003.1223481","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1223481","url":null,"abstract":"An improved version of the self-organization by lateral inhibition model (SOLI) has been applied to two synthetic data sets as well as the breast cancer and liver data sets, two well-known benchmark data sets. The methodology developed combines the use of various validity indices with the developed combines with the use of various validity indices with the SOLI model to discover the proper cluster structure within the data sets. In addition, the results explain why the breast cancer data set trends to be clustered so accurately while the liver data set trends to be so difficult to cluster accurately.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127150587","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jae-Byung Jung, J. Jacobs, G.A. Dowding, P. K. Simpson
{"title":"Initial species discrimination experiments with riverine salmonids","authors":"Jae-Byung Jung, J. Jacobs, G.A. Dowding, P. K. Simpson","doi":"10.1109/IJCNN.2003.1223882","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1223882","url":null,"abstract":"The primary objective of this study is to determine if there is sufficient information in broadband sonar echoes from the live tethered animals to determine their species. This study is focusing on two species: Chinook salmon (Oncorhynchus tshawytscha) and sockeye salmon (Oncorhynchus nerka). In addition to determining the species discrimination efficacy of the broadband sonar system, a comparison of these results with the seminal work of Burwen & Fleischman is made as they have presented a firm baseline for species discrimination in this riverine environment using a traditional narrowband split-beam hydroacoustic sonar system.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127354304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Coloring black boxes: visualization of neural network decisions","authors":"Wlodzislaw Duch","doi":"10.1109/IJCNN.2003.1223669","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1223669","url":null,"abstract":"Neural networks are commonly regarded as black boxes performing incomprehensible functions. For classification problems networks provide maps from high dimensional feature space to K-dimensional image space. Images of training vector are projected on polygon vertices, providing visualization of network function. Such visualization may show the dynamics of learning, allow for comparison of different networks, display training vectors around which potential problems may arise, show differences due to regularization and optimization procedures, investigate stability of network classification under perturbation of original vectors, and place new data sample in relation to training data, allowing for estimation of confidence in classification of a given sample. An illustrative example for the three-class Wine data and five-class Satimage data is described. The visualization method proposed here is applicable to any black box system that provides continuous outputs.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127466772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Automatic basis selection for RBF networks using Stein's unbiased risk estimator","authors":"A. Ghodsi, Dale Schuurmans","doi":"10.1109/IJCNN.2003.1223303","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1223303","url":null,"abstract":"The problem of selecting the appropriate number of basis functions is a critical issue for radial basis function neural networks. An RBF network with an overlay restricted basis gives poor predictions on new data, since the model has too little flexibility. By contrast, an RBF network with too many basis functions also gives poor generalization performance since it is too flexible and fits too much of the noise on the training data. Bias and variance are complementary quantities, and it is necessary to assign the number of basis function optimally in order to achieve the best compromise between them. In this paper we derive a theoretical criterion for assigning the appropriate number of basis functions. We use Stein's unbiased risk estimator (SURE) to drive a genetic criterion that defines the optimum number of basis functions to use for a given problem. The efficacy of this criterion is illustrated experimentally.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124824384","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A computational model of visual perception of surfaces","authors":"H. Eghbalnia, A. Assadi","doi":"10.1109/IJCNN.2003.1224031","DOIUrl":"https://doi.org/10.1109/IJCNN.2003.1224031","url":null,"abstract":"Models of processes underlying human perception of geometric forms are helpful in addressing the nature of computations performed by the brain in understanding images from environment. Perceptual geometry is an emerging field of interdisciplinary research whose objectives focus on study of geometry from the perspective of visual perception, and in turn, apply such geometric findings to the ecological study of vision. Perceptual geometry attempts to answer fundamental questions in perception of form and representation of space through synthesis of cognitive and biological theories of visual perception with geometric theories of the physical world. The following model appeals to perceptual geometry as a framework that takes advantage of \"almost regularity\" of 2-dimensional images of surfaces, in order to estimate of their shape, orientation and other geometric features.","PeriodicalId":310128,"journal":{"name":"Proceedings of the International Joint Conference on Neural Networks, 2003.","volume":"88 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2003-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123263880","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}