{"title":"Empirical evaluation of gradient methods for matrix learning vector quantization","authors":"Michael LeKander, Michael Biehl, Harm de Vries","doi":"10.1109/WSOM.2017.8020027","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020027","url":null,"abstract":"Generalized Matrix Learning Vector Quantization (GMLVQ) critically relies on the use of an optimization algorithm to train its model parameters. We test various schemes for automated control of learning rates in gradient-based training. We evaluate these algorithms in terms of their achieved performance and their practical feasibility. We find that some algorithms do indeed perform better than others across multiple benchmark datasets. These algorithms produce GMLVQ models which not only better fit the training data, but also perform better upon validation. In particular, we find that the Variance-based Stochastic Gradient Descent algorithm consistently performs best across all experiments.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"101 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115223803","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
T. Villmann, Michael Biehl, A. Villmann, S. Saralajew
{"title":"Fusion of deep learning architectures, multilayer feedforward networks and learning vector quantizers for deep classification learning","authors":"T. Villmann, Michael Biehl, A. Villmann, S. Saralajew","doi":"10.1109/WSOM.2017.8020009","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020009","url":null,"abstract":"The advantage of prototype based learning vector quantizers are the intuitive and simple model adaptation as well as the easy interpretability of the prototypes as class representatives for the class distribution to be learned. Although they frequently yield competitive performance and show robust behavior nowadays powerful alternatives have increasing attraction. Particularly, deep architectures of multilayer networks achieve frequently very high accuracies and are, thanks to modern graphic processor units use for calculation, trainable in acceptable time. In this conceptual paper we show, how we can combine both network architectures to benefit from their advantages. For this purpose, we consider learning vector quantizers in terms of feedforward network architectures and explain how it can be combined effectively with multilayer or single-layer feedforward network architectures. This approach includes deep and flat architectures as well as the popular extreme learning machines. For the resulting networks, the multi-/single-layer networks act as adaptive filters like in signal processing while the interpretability of the prototype-based learning vector quantizers is kept for the resulting filtered feature space. In this way a powerful combination of two successful architectures is obtained.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115557307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Straat, M. Kaden, M. Gay, T. Villmann, A. Lampe, U. Seiffert, Michael Biehl, F. Melchert
{"title":"Prototypes and matrix relevance learning in complex fourier space","authors":"M. Straat, M. Kaden, M. Gay, T. Villmann, A. Lampe, U. Seiffert, Michael Biehl, F. Melchert","doi":"10.1109/WSOM.2017.8020019","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020019","url":null,"abstract":"In this contribution, we consider the classification of time-series and similar functional data which can be represented in complex Fourier coefficient space. We apply versions of Learning Vector Quantization (LVQ) which are suitable for complex-valued data, based on the so-called Wirtinger calculus. It makes possible the formulation of gradient based update rules in the framework of cost-function based Generalized Matrix Relevance LVQ (GMLVQ). Alternatively, we consider the concatenation of real and imaginary parts of Fourier coefficients in a real-valued feature vector and the classification of time domain representations by means of conventional GMLVQ.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128030404","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Motivated self-organization","authors":"N. Rougier, Y. Boniface","doi":"10.1109/WSOM.2017.8020000","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020000","url":null,"abstract":"We present in this paper a variation of the self-organizing map algorithm where the original time-dependent (learning rate and neighborhood) learning function is replaced by a time-invariant one. The resulting self-organization does not fit the magnification law and the final vector density is not directly proportional to the density of the distribution. This lead us to introduce the notion of motivated self-organization where the self-organization is biased toward some data thanks to a supplementary signal. From a behavioral point of view, this signal may be understood as a motivational signal allowing a finer tuning of the final self-organization where needed. We illustrate this behavior through a simple robotic arm setup. Open access version of this article is available at https://hal.inria.fr/hal-01513519.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126063138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Credible visualizations for planar projections","authors":"A. Ultsch, Michael C. Thrun","doi":"10.1109/WSOM.2017.8020010","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020010","url":null,"abstract":"Planar projections, i.e. projections from a high dimensional data space onto a two dimensional plane, are still in use to detect structures, such as clusters, in multivariate data. It can be shown that only the subclass of focusing projections such as CCA, NeRV and the ESOM are able to disentangle linear non separable data. However, even these projections are sometimes erroneous. U-matrix methods are able to visualize these errors for SOM based projections. This paper extends the U-matrix methods to other projections in form of a so called generalized U-matrix. Based on previous work, an algorithm for the construction of generalized U-matrix is introduced, that is more efficient and free of parameters which may be hard to determine. Results are presented on a difficult artificial data set and a real word multivariate data set from cancer research.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133295832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Mulders, Cyril de Bodt, Johannes Bjelland, A. Pentland, M. Verleysen, Yves-Alexandre de Montjoye
{"title":"Improving individual predictions using social networks assortativity","authors":"D. Mulders, Cyril de Bodt, Johannes Bjelland, A. Pentland, M. Verleysen, Yves-Alexandre de Montjoye","doi":"10.1109/WSOM.2017.8020023","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020023","url":null,"abstract":"Social networks are known to be assortative with respect to many attributes, such as age, weight, wealth, level of education, ethnicity and gender. This can be explained by influences and homophilies. Independently of its origin, this assortativity gives us information about each node given its neighbors. Assortativity can thus be used to improve individual predictions in a broad range of situations, when data are missing or inaccurate. This paper presents a general framework based on probabilistic graphical models to exploit social network structures for improving individual predictions of node attributes. Using this framework, we quantify the assortativity range leading to an accuracy gain in several situations. We finally show how specific characteristics of the network can improve performances further. For instance, the gender assortativity in real-world mobile phone data changes significantly according to some communication attributes. In this case, individual predictions with 75% accuracy are improved by up to 3%.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"56 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124306036","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An evolutionary building algorithm for Deep Neural Networks","authors":"R. Zemouri","doi":"10.1109/WSOM.2017.8020002","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020002","url":null,"abstract":"The increase of the computer power has contributed significantly to the development of the Deep Neural Networks. However, the training phase is more difficult since there are many hidden layers with many connections. The aim of this paper is to improve the learning procedure for Deep Neural Networks. A new method for building an evolutionary DNN is presented. With our method, the user does not have to arbitrary specify the number of hidden layers nor the number of neurons per layer. Illustrative examples are provided to support the theoretical analysis.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115138726","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. C. Carneiro, Dayana Niazabeth Del Valle Silva Yanez, C. Ulsen, S. Fraser, Juliana Livi Antoniassi, S. Paz, R. Angélica, H. Kahn
{"title":"Imputation of reactive silica and available alumina in bauxites by self-organizing maps","authors":"C. C. Carneiro, Dayana Niazabeth Del Valle Silva Yanez, C. Ulsen, S. Fraser, Juliana Livi Antoniassi, S. Paz, R. Angélica, H. Kahn","doi":"10.1109/WSOM.2017.8020008","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020008","url":null,"abstract":"Geochemical analyses can provide multiple analytical variables. Accordingly, the generation of large geochemical databases enables imputation studies or analytical estimates of missing values or complex measuring. The processing of bauxite is a key step in the production of aluminum, in which the determination of Reactive Silica (RxSiO<inf>2</inf>) and Available Alumina (AvAl<inf>2</inf>O<inf>3</inf>) are very relevant. The traditional analytical method for achieving RxSiO<inf>2</inf> has limitations associated with poor repeatability and reproducibility of results. Based on the values from the unsupervised Self-Organizing Maps technique, this study aims to develop, systematically, the imputation of missing grades of the geochemical composition of bauxite samples of a database from three trial projects, for the variables: total Al<inf>2</inf>O<inf>3</inf>; total SiO<inf>2</inf>; total Fe<inf>2</inf>O<inf>3</inf>; and total TiO<inf>2</inf>. Each project was submitted to partial exclusion of AvAl<inf>2</inf>O<inf>3</inf> and RxSiO<inf>2</inf> values, in proportion of 20%, 30%, 40% and 50%, to investigate the SOM technique as imputation method for RxSiO<inf>2</inf> and AvAl<inf>2</inf>O<inf>3</inf>. By comparing the imputed values from the SOM analysis with the original values, SOM technique demonstrated to be an imputation tool capable of obtaining analytical results with up to 50% of missing data. Specifically, the best results demonstrate that AvAl<inf>2</inf>O<inf>3</inf> can be obtained by imputation with a higher correlation than RxSiO<inf>2</inf>, based on the parameters and variables involved in the study. Similarity in the nature of samples and an increase in the number of embedded analytical variables are factors that provided better imputation results.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"50 15","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"113934161","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"SOM-empowered graph segmentation for fast automatic clustering of large and complex data","authors":"E. Merényi, Joshua Taylor","doi":"10.1109/WSOM.2017.8020004","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020004","url":null,"abstract":"Many clustering methods, including modern graph segmentation algorithms, run into limitations when encountering “Big Data”, data with high feature dimensions, large volume, and complex structure. SOM-based clustering has been demonstrated to accurately capture many clusters of widely varying statistical properties in such data. While a number of automated SOM segmentations have been put forward, the best identifications of complex cluster structures to date are those performed interactively from informative visualizations of the learned SOM's knowledge. This does not scale for Big Data, large archives or near-real time analyses for fast decision-making. We present a new automated approach to SOM-segmentation which closely approximates the precision of the interactive method for complicated data, and at the same time is very fast and memory-efficient. We achieve this by infusing SOM knowledge into leading graph segmentation algorithms which, by themselves, produce extremely poor results segmenting the SOM prototypes. We use the SOM prototypes as input vectors and CONN similarity measure, derived from the SOM's knowledge of the data connectivity, as edge weighting to the graph segmentation algorithms. We demonstrate the effectiveness on synthetic data and on real spectral imagery.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134153102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Self-organizing map for orienteering problem with dubins vehicle","authors":"J. Faigl","doi":"10.1109/WSOM.2017.8020017","DOIUrl":"https://doi.org/10.1109/WSOM.2017.8020017","url":null,"abstract":"This paper reports on the application of the self-organizing map (SOM) to solve a novel generalization of the Orienteering Problem (OP) for curvature-constrained vehicles that is called the Dubins Orienteering Problem (DOP). Having a set of target locations, each with associated reward, and a given travel budget, the problem is to find the most valuable curvature-constrained path connecting the target locations such that the path does not exceed the travel budget. The proposed approach is based on two existing SOM-based approaches to solving the OP and Dubins Traveling Salesman Problem (Dubins TSP) that are further generalized to provide a solution of the more computational challenging DOP. DOP combines challenges of the combinatorial optimization of the OP and TSP to determine a subset of the most valuable targets and the optimal sequence of the waypoints to collect rewards of the targets together with the continuous optimization of determining headings of Dubins vehicle at the waypoints such that the total length of the curvature-constrained path is shorter than the given travel budget and the total sum of the collected rewards is maximized.","PeriodicalId":130086,"journal":{"name":"2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM)","volume":"124 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115449697","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}