NEUTRAMS: Neural network transformation and co-design under neuromorphic hardware constraints

2016 ◽  
Author(s):  
Yu Ji ◽  
YouHui Zhang ◽  
ShuangChen Li ◽  
Ping Chi ◽  
CiHang Jiang ◽  
...  
Keyword(s):  
Neural Network ◽  
Neuromorphic Hardware ◽  
Network Transformation

scholarly journals Memristor Neural Network Training with Clock Synchronous Neuromorphic System

Micromachines ◽  
2019 ◽  
Vol 10 (6) ◽  
pp. 384 ◽  
Author(s):  
Sumin Jo ◽  
Wookyung Sun ◽  
Bokyung Kim ◽  
Sunhee Kim ◽  
Junhee Park ◽  
...  
Keyword(s):  
Neural Network ◽  
Unsupervised Learning ◽  
Spike Timing ◽  
Machine Learning Algorithms ◽  
Training Methods ◽  
Nonlinear Characteristic ◽  
Hardware System ◽  
Neuromorphic Hardware ◽  
Mathematical Formulas ◽  
Inputs And Outputs

Memristor devices are considered to have the potential to implement unsupervised learning, especially spike timing-dependent plasticity (STDP), in the field of neuromorphic hardware research. In this study, a neuromorphic hardware system for multilayer unsupervised learning was designed, and unsupervised learning was performed with a memristor neural network. We showed that the nonlinear characteristic memristor neural network can be trained by unsupervised learning only with the correlation between inputs and outputs. Moreover, a method to train nonlinear memristor devices in a supervised manner, named guide training, was devised. Memristor devices have a nonlinear characteristic, which makes implementing machine learning algorithms, such as backpropagation, difficult. The guide-training algorithm devised in this paper updates the synaptic weights by only using the correlations between inputs and outputs, and therefore, neither complex mathematical formulas nor computations are required during the training. Thus, it is considered appropriate to train a nonlinear memristor neural network. All training and inference simulations were performed using the designed neuromorphic hardware system. With the system and memristor neural network, the image classification was successfully done using both the Hebbian unsupervised training and guide supervised training methods.

scholarly journals A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems

2017 ◽  
Vol 7 (1) ◽  
Author(s):  
Marc Osswald ◽  
Sio-Hoi Ieng ◽  
Ryad Benosman ◽  
Giacomo Indiveri
Keyword(s):  
Neural Network ◽  
Machine Vision ◽  
Stereo Vision ◽  
Correspondence Problem ◽  
Spiking Neural Network ◽  
Neuromorphic Engineering ◽  
Stereo Correspondence ◽  
Vision Systems ◽  
Neuromorphic Hardware ◽  
Event Based

Abstract Stereo vision is an important feature that enables machine vision systems to perceive their environment in 3D. While machine vision has spawned a variety of software algorithms to solve the stereo-correspondence problem, their implementation and integration in small, fast, and efficient hardware vision systems remains a difficult challenge. Recent advances made in neuromorphic engineering offer a possible solution to this problem, with the use of a new class of event-based vision sensors and neural processing devices inspired by the organizing principles of the brain. Here we propose a radically novel model that solves the stereo-correspondence problem with a spiking neural network that can be directly implemented with massively parallel, compact, low-latency and low-power neuromorphic engineering devices. We validate the model with experimental results, highlighting features that are in agreement with both computational neuroscience stereo vision theories and experimental findings. We demonstrate its features with a prototype neuromorphic hardware system and provide testable predictions on the role of spike-based representations and temporal dynamics in biological stereo vision processing systems.

Neuromorphic Memristive Chips: Design and Technology

2021 ◽  
Vol 23 (6) ◽  
pp. 285-294
Author(s):  
N.V. Andreeva ◽  
◽  
V.V. Luchinin ◽  
E.A. Ryndin ◽  
M.G. Anchkov ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Resistive Switching ◽  
Hardware Implementation ◽  
Functional Materials ◽  
Design And Technology ◽  
Spiking Neural Network ◽  
Chip Design ◽  
Neuromorphic Hardware ◽  
Metal Insulator Metal

Memristive neuromorphic chips exploit a prospective class of novel functional materials (memristors) to deploy a new architecture of spiking neural networks for developing basic blocks of brain-like systems. Memristor-based neuromorphic hardware solutions for multi-agent systems are considered as challenges in frontier areas of chip design for fast and energy-efficient computing. As functional materials, metal oxide thin films with resistive switching and memory effects (memristive structures) are recognized as a potential elemental base for new components of neuromorphic engineering, enabling a combination of both data storage and processing in a single unit. A key design issue in this case is a hardware defined functionality of neural networks. The gradient change of resistive properties of memristive elements and its non-volatile memory behavior ensure the possibility of spiking neural network organization with unsupervised learning through hardware implementation of basic synaptic mechanisms, such as Hebb's learning rules including spike — timing dependent plasticity, long-term potentiation and depression. This paper provides an overview of scientific researches carrying out at Saint Petersburg Electrotechnical University "LETI" since 2014 in the field of novel electronic components for neuromorphic hardware solutions of brain-like chip design. Among the most promising concepts developed by ETU "LETI" are: the design of metal-insulator-metal structures exhibiting multilevel resistive switching (gradient tuning of resistive properties and bipolar resistive switching are combined together in a sin¬gle memristive element) for further use as artificial synaptic devices in neuromorphic chips; computing schemes for spatio-temporal pattern recognition based on spiking neural network architecture implementation; breadboard models of analogue circuits of hardware implementation of neuromorphic blocks for brain-like system developing.

scholarly journals Bitstream-Based Neural Network for Scalable, Efficient, and Accurate Deep Learning Hardware

2020 ◽  
Vol 14 ◽  
Author(s):  
Hyeonuk Sim ◽  
Jongeun Lee
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recognition Accuracy ◽  
Fault Tolerant ◽  
State Of The Art ◽  
The Other ◽  
Stochastic Computing ◽  
Hardware Implementations ◽  
Neuromorphic Hardware ◽  
Very High

While convolutional neural networks (CNNs) continue to renew state-of-the-art performance across many fields of machine learning, their hardware implementations tend to be very costly and inflexible. Neuromorphic hardware, on the other hand, targets higher efficiency but their inference accuracy lags far behind that of CNNs. To bridge the gap between deep learning and neuromorphic computing, we present bitstream-based neural network, which is both efficient and accurate as well as being flexible in terms of arithmetic precision and hardware size. Our bitstream-based neural network (called SC-CNN) is built on top of CNN but inspired by stochastic computing (SC), which uses bitstreams to represent numbers. Being based on CNN, our SC-CNN can be trained with backpropagation, ensuring very high inference accuracy. At the same time our SC-CNN is deterministic, hence repeatable, and is highly accurate and scalable even to large networks. Our experimental results demonstrate that our SC-CNN is highly accurate up to ImageNet-targeting CNNs, and improves efficiency over conventional digital designs ranging through 50–100% in operations-per-area depending on the CNN and the application scenario, while losing <1% in recognition accuracy. In addition, our SC-CNN implementations can be much more fault-tolerant than conventional digital implementations.

scholarly journals Real-time cortical simulation on neuromorphic hardware

2019 ◽  
Vol 378 (2164) ◽  
pp. 20190160 ◽  
Author(s):  
Oliver Rhodes ◽  
Luca Peres ◽  
Andrew G. D. Rowley ◽  
Andrew Gait ◽  
Luis A. Plana ◽  
...  
Keyword(s):  
Neural Network ◽  
Real Time ◽  
Large Scale ◽  
Spiking Neural Network ◽  
Real Time Processing ◽  
Real Time Simulation ◽  
Statistical Measures ◽  
Time Simulation ◽  
Neuromorphic Hardware ◽  
Cortical Microcircuit

Real-time simulation of a large-scale biologically representative spiking neural network is presented, through the use of a heterogeneous parallelization scheme and SpiNNaker neuromorphic hardware. A published cortical microcircuit model is used as a benchmark test case, representing ≈1 mm 2 of early sensory cortex, containing 77 k neurons and 0.3 billion synapses. This is the first hard real-time simulation of this model, with 10 s of biological simulation time executed in 10 s wall-clock time. This surpasses best-published efforts on HPC neural simulators (3 × slowdown) and GPUs running optimized spiking neural network (SNN) libraries (2 × slowdown). Furthermore, the presented approach indicates that real-time processing can be maintained with increasing SNN size, breaking the communication barrier incurred by traditional computing machinery. Model results are compared to an established HPC simulator baseline to verify simulation correctness, comparing well across a range of statistical measures. Energy to solution and energy per synaptic event are also reported, demonstrating that the relatively low-tech SpiNNaker processors achieve a 10 × reduction in energy relative to modern HPC systems, and comparable energy consumption to modern GPUs. Finally, system robustness is demonstrated through multiple 12 h simulations of the cortical microcircuit, each simulating 12 h of biological time, and demonstrating the potential of neuromorphic hardware as a neuroscience research tool for studying complex spiking neural networks over extended time periods. This article is part of the theme issue ‘Harmonizing energy-autonomous computing and intelligence’.

Sign in / Sign up

Export Citation Format

Share Document