J. Aimone, Ojas D. Parekh, C. Phillips, A. Pınar, William M. Severa, Helen Xu
{"title":"Dynamic Programming with Spiking Neural Computing","authors":"J. Aimone, Ojas D. Parekh, C. Phillips, A. Pınar, William M. Severa, Helen Xu","doi":"10.1145/3354265.3354285","DOIUrl":"https://doi.org/10.1145/3354265.3354285","url":null,"abstract":"With the advent of large-scale neuromorphic platforms, we seek to better understand the applications of neuromorphic computing to more general-purpose computing domains. Graph analysis problems have grown increasingly relevant in the wake of readily available massive data. We demonstrate that a broad class of combinatorial and graph problems known as dynamic programs enjoy simple and efficient neuromorphic implementations, by developing a general technique to convert dynamic programs to spiking neuromorphic algorithms. Dynamic programs have been studied for over 50 years and have dozens of applications across many fields.","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131893533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Approximate Pattern Matching using Hierarchical Graph Construction and Sparse Distributed Representation","authors":"Aakanksha Mathuria, D. Hammerstrom","doi":"10.1145/3354265.3354286","DOIUrl":"https://doi.org/10.1145/3354265.3354286","url":null,"abstract":"With recent developments in deep networks, there have been significant advances in visual object detection and recognition. However, some of these networks are still easily fooled/hacked and have shown \"bag of features\" failures. Some of this is due to the fact that even deep networks make only marginal use of the complex structure that exists in real-world images, even after training on huge numbers of images. Biology appears to take advantage of such a structure, but how? In our research, we are studying approaches for robust pattern matching using still, 2D Blocks World images based on graphical representations of the various components of an image. Such higher order information represents the \"structure\" of the visual object. Here we discuss how the structural information of an image can be captured in a Sparse Distributed Representation (SDR) loosely based on cortical circuits. We apply probabilistic graph isomorphism and subgraph isomorphism to our 2D Blocks World images and achieve O (1) and O (nk) complexity for an approximate match. The optimal match is an NP-Hard problem. The image labeled graph is created using OpenCV to find the object contours and objects' labels and a fixed radius nearest neighbor algorithm to build the edges between the objects. Pattern matching is done using the properties of SDRs. Our research shows the promise of applying graph-based neuromorphic techniques for pattern matching of images based on such structure.","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124163731","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Design and Analysis of Real Time Spiking Neural Network Decoder for Neuromorphic Chips","authors":"Chenyuan Zhao, Lingjia Liu, Y. Yi","doi":"10.1145/3354265.3354280","DOIUrl":"https://doi.org/10.1145/3354265.3354280","url":null,"abstract":"Neuromorphic computing, which is based on non-traditional architectures that mimic bio-neurological process, could offer potentially disruptive capabilities and high energy-efficiency in real-time data monitoring and prediction, and resource allocation. The first step for de-signing neuromorphic chips is to explore effective design methodologies for real time and energy efficient spiking neural network (SNN) encoders and decoders. In this paper, a spike timing dependent plasticity principle (STDP) based decoder is designed and optimized. As shown in our experimental results, the proposed STDP based SNN decoder achieves good performance in information recovery with multiple scales.","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124773838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"MLP+NeuroSimV3.0: Improving On-chip Learning Performance with Device to Algorithm Optimizations","authors":"Yandong Luo, Xiaochen Peng, Shimeng Yu","doi":"10.1145/3354265.3354266","DOIUrl":"https://doi.org/10.1145/3354265.3354266","url":null,"abstract":"On-chip learning with compute-in-memory (CIM) paradigm has become popular in machine learning hardware design in the recent years. However, it is hard to achieve high on-chip learning accuracy due to the high nonlinearity in the weight update curve of emerging nonvolatile memory (eNVM) based analog synapse devices. Although digital synapse devices offer good learning accuracy, the row-by-row partial sum accumulation leads to high latency. In this paper, the methods to solve the aforementioned issues are presented with a device-to-algorithm level optimization. For analog synapses, novel hybrid precision synapses with good linearity and more advanced training algorithms are introduced to increase the on-chip learning accuracy. The latency issue for digital synapses can be solved by using parallel partial sum read-out scheme. All these features are included into the recently released MLP + NeuroSimV3.0, which is an in-house developed device-to-system evaluation framework for neuro-inspired accelerators based on CIM paradigm.","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"125 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131650101","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Feature Extraction using Spiking Convolutional Neural Networks","authors":"Ruthvik Vaila, John N. Chiasson, V. Saxena","doi":"10.1145/3354265.3354279","DOIUrl":"https://doi.org/10.1145/3354265.3354279","url":null,"abstract":"Spiking neural networks are biologically plausible counterparts of the artificial neural networks. Conventional (non spiking) artificial neural networks are trained using a stochastic gradient descent algorithm (back propagation) while spiking neural networks are trained using the spike timing dependant plasticity algorithm. Training conventional neural networks is a memory and computationally/energy intensive job. Spiking networks show the potential to reduce both memory and energy usage. In this work, we consider spiking neural networks (SNN) as a classifier of the MNIST and NM-NIST digits. We compare SNNs with convolutional neural networks in terms of classification accuracy, catastrophic forgetting.","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116432407","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Younes Bouhadjar, M. Diesmann, R. Waser, D. Wouters, T. Tetzlaff
{"title":"Constraints on sequence processing speed in biological neuronal networks","authors":"Younes Bouhadjar, M. Diesmann, R. Waser, D. Wouters, T. Tetzlaff","doi":"10.1145/3354265.3354281","DOIUrl":"https://doi.org/10.1145/3354265.3354281","url":null,"abstract":"Sequence processing has been proposed to be the universal computation performed by the neocortex. The Hierarchical Temporal Memory (HTM) model provides a mechanistic implementation of this form of processing. While the model accounts for a number of neocortical features, it is based on networks of highly abstract neuron and synapse models updated in discrete time. Here, we reformulate the model in terms of a network of spiking neurons with continuous-time dynamics to investigate how neuronal parameters such as cell-intrinsic time constants and synaptic weights constrain the sequence-processing speed.","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130867497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the International Conference on Neuromorphic Systems","authors":"","doi":"10.1145/3354265","DOIUrl":"https://doi.org/10.1145/3354265","url":null,"abstract":"","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126928612","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Approximating Back-propagation for a Biologically Plausible Local Learning Rule in Spiking Neural Networks","authors":"Amar Shrestha, Haowen Fang, Qing Wu, Qinru Qiu","doi":"10.1145/3354265.3354275","DOIUrl":"https://doi.org/10.1145/3354265.3354275","url":null,"abstract":"Asynchronous event-driven computation and communication using spikes facilitate the realization of spiking neural networks (SNN) to be massively parallel, extremely energy efficient and highly robust on specialized neuromorphic hardware. However, the lack of a unified robust learning algorithm limits the SNN to shallow networks with low accuracies. Artificial neural networks (ANN), however, have the backpropagation algorithm which can utilize gradient descent to train networks which are locally robust universal function approximators. But backpropagation algorithm is neither biologically plausible nor neuromorphic implementation friendly because it requires: 1) separate backward and forward passes, 2) differentiable neurons, 3) high-precision propagated errors, 4) coherent copy of weight matrices at feedforward weights and the backward pass, and 5) non-local weight update. Thus, we propose an approximation of the backpropagation algorithm completely with spiking neurons and extend it to a local weight update rule which resembles a biologically plausible learning rule spike-timing-dependent plasticity (STDP). This will enable error propagation through spiking neurons for a more biologically plausible and neuromorphic implementation friendly backpropagation algorithm for SNNs. We test the proposed algorithm on various traditional and non-traditional benchmarks with competitive results.","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"54 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126850153","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Training Spiking Networks via Natural Evolution Strategies","authors":"W. Olin-Ammentorp, N. Cady","doi":"10.1145/3354265.3354283","DOIUrl":"https://doi.org/10.1145/3354265.3354283","url":null,"abstract":"Approaching the training of spiking neural networks as a black-box optimization problem, we compare the performance of two evolutionary methods which must train the weights of topologically-fixed spiking networks towards different tasks. One evolutionary method is a 'traditional' optimizer, and the other is a natural evolutionary strategy (NES) which attempts to estimate the fitness landscape to make a more direct ascent towards higher expected fitness values. We find that overall, the NES method can train networks faster than the traditional EO method for the given pole-balancing problem.","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129330950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Graph Partitioning as Quadratic Unconstrained Binary Optimization (QUBO) on Spiking Neuromorphic Hardware","authors":"S. Mniszewski","doi":"10.1145/3354265.3354269","DOIUrl":"https://doi.org/10.1145/3354265.3354269","url":null,"abstract":"In this work, graph partitioning (GP) is explored using quadratic unconstrained binary optimization (QUBO) on the IBM TrueNorth spiking neuromorphic architecture. GP splits a graph into similar-sized parts while minimizing the number of cut edges between parts. Classical approaches to GP rely on heuristics and approximation algorithms. The GP QUBO formulation was inspired by previous work using the D-Wave quantum annealer. This approach is not limited to graph algorithms, but is applicable to solving a spectrum of NP-hard optimization problems. A classical pseudo simulated annealing metaheuristic is used to solve the QUBO. Implementation on the IBM TrueNorth using a spiking framework is described. Results as converged high-energy solutions are shown to be \"good enough\" or optimal for partitioning a graph into 2 parts.","PeriodicalId":225636,"journal":{"name":"Proceedings of the International Conference on Neuromorphic Systems","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115441814","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}