{"title":"Transfer learning for Latin and Chinese characters with Deep Neural Networks","authors":"D. Ciresan, U. Meier, J. Schmidhuber","doi":"10.1109/IJCNN.2012.6252544","DOIUrl":"https://doi.org/10.1109/IJCNN.2012.6252544","url":null,"abstract":"We analyze transfer learning with Deep Neural Networks (DNN) on various character recognition tasks. DNN trained on digits are perfectly capable of recognizing uppercase letters with minimal retraining. They are on par with DNN fully trained on uppercase letters, but train much faster. DNN trained on Chinese characters easily recognize uppercase Latin letters. Learning Chinese characters is accelerated by first pretraining a DNN on a small subset of all classes and then continuing to train on all classes. Furthermore, pretrained nets consistently outperform randomly initialized nets on new tasks with few labeled data.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"215 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115979160","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Particle filters and beamforming for EEG source estimation","authors":"P. Georgieva, L. Mihaylova, N. Bouaynaya, L. Jain","doi":"10.1109/IJCNN.2012.6252516","DOIUrl":"https://doi.org/10.1109/IJCNN.2012.6252516","url":null,"abstract":"This is a proof of concept work that proposes a solution to the inverse problem of EEG source estimation by combining two techniques, namely a Particle Filter (PF) for geometrical (3D) localization of the most active brain zones (expressed by two dipoles) and a beamformer (BF) as a spatial filter for estimation of the oscillations that have originated the recorded EEG data. The estimation is reliable for uncorrelated brain sources.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"170 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116602232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Khushaba, S. Kodagoda, G. Dissanayake, Luke Greenacre, Sandra Burke, J. Louviere
{"title":"A neuroscientific approach to choice modeling: Electroencephalogram (EEG) and user preferences","authors":"R. Khushaba, S. Kodagoda, G. Dissanayake, Luke Greenacre, Sandra Burke, J. Louviere","doi":"10.1109/IJCNN.2012.6252561","DOIUrl":"https://doi.org/10.1109/IJCNN.2012.6252561","url":null,"abstract":"Discrete choice experiments have traditionally focused on improving the prediction of static choices that are measured through external reflection and surveys. It is argued that considering the underlying processes of decision making across a variety of contexts may further progress decision research. As a pilot study in this field, this paper explores the dynamic nature of decision-making by examining the associated brain activity, Electroencephalogram (EEG), of people while undertaking choices designed to elicit their preferences. To facilitate such a study, the Tobii-Studio eye tracker system was utilized to capture the participants' choice based preferences when they were observing seventy two sets of objects of three images offering potential personal computer backgrounds. Choice based preferences were identified by having the respondent click on their preferred image. In addition, the commercial Emotiv wireless EEG headset with 14 channels was utilized to capture the associated brain activity during the period of the experiments. Principal Component Analysis (PCA) was utilized to preprocess the EEG data before analyzing it with the Fast Fourier Transform (FFT) to observe the changes in the four principal frequency bands, theta (3 - 7 Hz), alpha (8 - 12 Hz), beta (13 - 30 Hz), and gamma (30 - 40 Hz). A mutual information (MI) measure was then used to study left-to-right hemisphere differences as well as front-to-back difference. Across six recruited participants there was a clear and significant change in the spectral activities taking place mainly in the frontal (theta and alpha across F3 and F4) and occipital (alpha and beta across O1 and O2) regions while the participants were indicating their preferences.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116603077","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Predicting protein-ligand binding site with differential evolution and support vector machine","authors":"Ginny Y. Wong, Frank H. F. Leung, Sai Ho Ling","doi":"10.1109/ijcnn.2012.6252744","DOIUrl":"https://doi.org/10.1109/ijcnn.2012.6252744","url":null,"abstract":"Identification of protein-ligand binding site is an important task in structure-based drug design and docking algorithms. In these two decades, many different approaches have been developed to predict the binding site, such as geometric, energetic and sequence-based methods. When the scores are calculated from these methods, the method of classification is very important and can affect the prediction results greatly. A developed support vector machine (SVM) is used to classify the pockets, which are most likely to bind ligands with the attributes of grid value, interaction potential, offset from protein, conservation score and the information around the pockets. Since SVM is sensitive to the input parameters and the positive samples are more relevant than negative samples, differential evolution (DE) is applied to find out the suitable parameters for SVM. We compare our algorithm to four other approaches: LIGSITE, SURFNET, PocketFinder and Concavity. Our algorithm is found to provide the highest success rate.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126975489","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Classifying malignant brain tumours from 1H-MRS data using Breadth Ensemble Learning","authors":"A. Vilamala, L. B. Muñoz, A. Vellido","doi":"10.1109/IJCNN.2012.6252756","DOIUrl":"https://doi.org/10.1109/IJCNN.2012.6252756","url":null,"abstract":"In neuro oncology, the accurate diagnostic identification and characterization of tumours is paramount for determining their prognosis and the adequate course of treatment. This is usually a difficult problem per se, due to the localization of the tumour in an extremely sensitive and difficult to reach organ such as the brain. The clinical analysis of brain tumours often requires the use of non-invasive measurement methods, the most common of which resort to imaging techniques. The discrimination between high-grade malignant tumours of different origin but similar characteristics, such as glioblastomas and metastases, is a particularly difficult problem in this context. This is because imaging techniques are often not sensitive enough and their spectroscopic signal is overall too similar. In spite of this, machine learning techniques, coupled with robust feature selection procedures, have recently made substantial inroads into the problem. In this study, magnetic resonance spectroscopy data from an international, multi-centre database were used to discriminate between these two types of malignant brain tumours using ensemble learning techniques, with a focus on the definition of a feature selection method specifically designed for ensembles. This method, Breadth Ensemble Learning, takes advantage of the fact that many of the frequencies of the available spectra convey no relevant information for the discrimination of the tumours. The potential of the proposed method is supported by some of the best results reported to date for this problem.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121856198","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A spectral algorithm for topographical Co-clustering","authors":"Nicoleta Rogovschi, Lazhar Labiod, M. Nadif","doi":"10.1109/IJCNN.2012.6252398","DOIUrl":"https://doi.org/10.1109/IJCNN.2012.6252398","url":null,"abstract":"This paper proposes a spectral algorithm for cross-topographic clustering. It leads to a simultaneous clustering on the rows and columns of data matrix, as well as the projection of the clusters on a two-dimensional grid while preserving the topological order of the initial data. The proposed algorithm is based on a spectral decomposition of this data matrix and the definition of a new matrix taking into account the co-clustering problem. The proposed approach has been validated on multiple datasets and the experimental results have shown very promising performance.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"66 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133503869","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Deep, super-narrow neural network is a universal classifier","authors":"Lech Szymanski, B. McCane","doi":"10.1109/IJCNN.2012.6252513","DOIUrl":"https://doi.org/10.1109/IJCNN.2012.6252513","url":null,"abstract":"Deep architecture models are known to be conducive to good generalisation for certain types of classification tasks. Existing unsupervised and semi-supervised training methods do not explain why and when deep internal representations will be effective. We investigate the fundamental principles of representation in deep architectures by devising a method for binary classification in multi-layer feed forward networks with limited breadth. We show that, given enough layers, a super-narrow neural network, with two neurons per layer, is capable of shattering any separable binary dataset. We also show that datasets that exhibit certain type of symmetries are better suited for deep representation and may require only few hidden layers to produce desired classification.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133938746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Analysis of pedestrian spatial behaviour using GDTW-P-SVMs","authors":"A. Jalalian, S. Chalup, Michael J. Ostwald","doi":"10.1109/IJCNN.2012.6252584","DOIUrl":"https://doi.org/10.1109/IJCNN.2012.6252584","url":null,"abstract":"This paper presents an analysis system to find the impact of architectural designs on pedestrian behavioural data. The system employs GDTW-P-SVMs which are capable of modelling sequential data with variable-length input series. We apply GDTW-P-SVMs to simulated pedestrian spatial behaviour data. The data include four types of behavioural characteristics: i) movement trajectories, ii) walking speed, iii) the angle α between the movement vector and the gaze vector and iv) its derivative. The analysis system learns a statistical model characterising three classes of spatial behaviour. The classes are formed based on pedestrians' reactions to visual attractions in a simulated environment. A separate data set that includes the crowd attraction effect is used to discuss the impact of social group formation on the classification result. Our experiments show that using the angle α and its derivative as input to the classifiers results in lower classification error rates compared to classification of trajectory and speed of movement data. We compare the classification accuracy of the GDTW-P-SVMs with other classification methods that are capable of handling data objects with variable-length input series. GDTW-P-SVMs showed promising results in classifying the simulated behavioural data.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134303619","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Autonomous learning of collaboration among robots","authors":"P. Arena, L. Patané, A. Vitanza","doi":"10.1109/IJCNN.2012.6252664","DOIUrl":"https://doi.org/10.1109/IJCNN.2012.6252664","url":null,"abstract":"The aim of this paper is to study the emergence of coordinated activities, and the investigation of collaboration between individuals in a small group of robots. The idea is to impose very simple global rules and to give a primary role to the environment mediation. In the paper the specialization strategy, already introduced in a previous work is extended, to autonomously solve a task assignment problem among agents in an initially homogeneous swarm. In particular, a given sequence of tasks is assigned to the group and each robot has to autonomously specialise in solving sub-sequences, resulting in a labor division which improves the performance of the team. Behavioral improvement is guided by a global reward function. Results, obtained in a dynamic simulation environment, show that performances depend by environmental conditions and starting positions of the singular agents: environment and the other robots play clearly a fundamental role in mediating the swarm capabilities.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"74 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134411030","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Data driven fitting sample selection for time series forecasting with neural networks","authors":"N. Kourentzes","doi":"10.1109/IJCNN.2012.6252528","DOIUrl":"https://doi.org/10.1109/IJCNN.2012.6252528","url":null,"abstract":"In this paper we propose a data driven method to select the fitting sample of neural networks for time series forecasting. In spite of the fundamental importance of sample selection for model building there has been limited research in the forecasting literature, mostly concluding in vague recommendations on how much time series history should be used and stored. This research addresses this issue in a data driven framework. The proposed method allows the neural networks to iteratively adjust the fitting sample, penalizing the time series history for age and inconsistent behavior. The resulting selected sample helps the networks to produce accurate out-of-sample forecasts, focusing on the recent history of the time series. The performance of the method is demonstrated using time series from different domains, exhibiting substantial improvements in accuracy.","PeriodicalId":287844,"journal":{"name":"The 2012 International Joint Conference on Neural Networks (IJCNN)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133840994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}