scholarly journals Memristor Neural Network Training with Clock Synchronous Neuromorphic System

Micromachines ◽  
2019 ◽  
Vol 10 (6) ◽  
pp. 384 ◽  
Author(s):  
Sumin Jo ◽  
Wookyung Sun ◽  
Bokyung Kim ◽  
Sunhee Kim ◽  
Junhee Park ◽  
...  
Keyword(s):  
Neural Network ◽  
Unsupervised Learning ◽  
Spike Timing ◽  
Machine Learning Algorithms ◽  
Training Methods ◽  
Nonlinear Characteristic ◽  
Hardware System ◽  
Neuromorphic Hardware ◽  
Mathematical Formulas ◽  
Inputs And Outputs

Memristor devices are considered to have the potential to implement unsupervised learning, especially spike timing-dependent plasticity (STDP), in the field of neuromorphic hardware research. In this study, a neuromorphic hardware system for multilayer unsupervised learning was designed, and unsupervised learning was performed with a memristor neural network. We showed that the nonlinear characteristic memristor neural network can be trained by unsupervised learning only with the correlation between inputs and outputs. Moreover, a method to train nonlinear memristor devices in a supervised manner, named guide training, was devised. Memristor devices have a nonlinear characteristic, which makes implementing machine learning algorithms, such as backpropagation, difficult. The guide-training algorithm devised in this paper updates the synaptic weights by only using the correlations between inputs and outputs, and therefore, neither complex mathematical formulas nor computations are required during the training. Thus, it is considered appropriate to train a nonlinear memristor neural network. All training and inference simulations were performed using the designed neuromorphic hardware system. With the system and memristor neural network, the image classification was successfully done using both the Hebbian unsupervised training and guide supervised training methods.

scholarly journals Stochastic binary synapses having sigmoidal cumulative distribution functions for unsupervised learning with spike timing-dependent plasticity

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yoshifumi Nishi ◽  
Kumiko Nomura ◽  
Takao Marukame ◽  
Koichi Mizushima
Keyword(s):  
Unsupervised Learning ◽  
Recognition Accuracy ◽  
Learning Rule ◽  
Spike Timing ◽  
Distribution Functions ◽  
Cumulative Distribution ◽  
Cumulative Probability ◽  
Spike Timing Dependent Plasticity ◽  
Dependent Plasticity ◽  
Neuromorphic Hardware

AbstractSpike timing-dependent plasticity (STDP), which is widely studied as a fundamental synaptic update rule for neuromorphic hardware, requires precise control of continuous weights. From the viewpoint of hardware implementation, a simplified update rule is desirable. Although simplified STDP with stochastic binary synapses was proposed previously, we find that it leads to degradation of memory maintenance during learning, which is unfavourable for unsupervised online learning. In this work, we propose a stochastic binary synaptic model where the cumulative probability of the weight change evolves in a sigmoidal fashion with potentiation or depression trials, which can be implemented using a pair of switching devices consisting of serially connected multiple binary memristors. As a benchmark test we perform simulations of unsupervised learning of MNIST images with a two-layer network and show that simplified STDP in combination with this model can outperform conventional rules with continuous weights not only in memory maintenance but also in recognition accuracy. Our method achieves 97.3% in recognition accuracy, which is higher than that reported with standard STDP in the same framework. We also show that the high performance of our learning rule is robust against device-to-device variability of the memristor's probabilistic behaviour.

scholarly journals Unsupervised Learning Method for SAR Image Classification Based on Spiking Neural Network

2021 ◽  
Author(s):  
Jiankun Chen ◽  
Xiaolan Qiu ◽  
Chuanzhao Han ◽  
Yirong Wu
Keyword(s):  
Neural Network ◽  
Unsupervised Learning ◽  
Image Classification ◽  
Spatial Information ◽  
Learning Algorithm ◽  
Image Interpretation ◽  
Spike Timing ◽  
Spiking Neural Network ◽  
Sar Image ◽  
The Brain

Recent neuroscience research results show that the nerve information in the brain is not only encoded by the spatial information. Spiking neural network based on pulse frequency coding plays a very important role in dealing with the problem of brain signal, especially complicated space-time information. In this paper, an unsupervised learning algorithm for bilayer feedforward spiking neural networks based on spike-timing dependent plasticity (STDP) competitiveness is proposed and applied to SAR image classification on MSTAR for the first time. The SNN learns autonomously from the input value without any labeled signal and the overall classification accuracy of SAR targets reached 80.8%. The experimental results show that the algorithm adopts the synaptic neurons and network structure with stronger biological rationality, and has the ability to classify targets on SAR image. Meanwhile, the feature map extraction ability of neurons is visualized by the generative property of SNN, which is a beneficial attempt to apply the brain-like neural network into SAR image interpretation.

Improving the effectiveness of traditional education based on computer artificial intelligence and neural network system

2020 ◽  
pp. 1-11
Author(s):  
Wenjuan Ma ◽  
Xuesi Zhao ◽  
Yuxiu Guo
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Education Reform ◽  
Traditional Education ◽  
Machine Learning Algorithms ◽  
Local Optimum ◽  
Optimization Strategy ◽  
Fish School ◽  
Tlbo Algorithm ◽  
Neural Network System

The application of artificial intelligence and machine learning algorithms in education reform is an inevitable trend of teaching development. In order to improve the teaching intelligence, this paper builds an auxiliary teaching system based on computer artificial intelligence and neural network based on the traditional teaching model. Moreover, in this paper, the optimization strategy is adopted in the TLBO algorithm to reduce the running time of the algorithm, and the extracurricular learning mechanism is introduced to increase the adjustable parameters, which is conducive to the algorithm jumping out of the local optimum. In addition, in this paper, the crowding factor in the fish school algorithm is used to define the degree or restraint of teachers’ control over students. At the same time, students in the crowded range gather near the teacher, and some students who are difficult to restrain perform the following behavior to follow the top students. Finally, this study builds a model based on actual needs, and designs a control experiment to verify the system performance. The results show that the system constructed in this paper has good performance and can provide a theoretical reference for related research.

scholarly journals Data-Driven Stability Assessment of Multilayer Long Short-Term Memory Networks

2021 ◽  
Vol 11 (4) ◽  
pp. 1829
Author(s):  
Davide Grande ◽  
Catherine A. Harris ◽  
Giles Thomas ◽  
Enrico Anderlini
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Short Term Memory ◽  
Moving Average ◽  
Machine Learning Algorithms ◽  
Physical Models ◽  
Data Driven ◽  
The Stability ◽  
And Control ◽  
Nonlinear Autoregressive

Recurrent Neural Networks (RNNs) are increasingly being used for model identification, forecasting and control. When identifying physical models with unknown mathematical knowledge of the system, Nonlinear AutoRegressive models with eXogenous inputs (NARX) or Nonlinear AutoRegressive Moving-Average models with eXogenous inputs (NARMAX) methods are typically used. In the context of data-driven control, machine learning algorithms are proven to have comparable performances to advanced control techniques, but lack the properties of the traditional stability theory. This paper illustrates a method to prove a posteriori the stability of a generic neural network, showing its application to the state-of-the-art RNN architecture. The presented method relies on identifying the poles associated with the network designed starting from the input/output data. Providing a framework to guarantee the stability of any neural network architecture combined with the generalisability properties and applicability to different fields can significantly broaden their use in dynamic systems modelling and control.

Design of a Robust Memristive Spiking Neuromorphic System with Unsupervised Learning in Hardware

2021 ◽  
Vol 17 (4) ◽  
pp. 1-26
Author(s):  
Md Musabbir Adnan ◽  
Sagarvarma Sayyaparaju ◽  
Samuel D. Brown ◽  
Mst Shamim Ara Shawkat ◽  
Catherine D. Schuman ◽  
...  
Keyword(s):  
Unsupervised Learning ◽  
Recognition Task ◽  
Single Layer ◽  
Process Variations ◽  
Spike Timing ◽  
System Level ◽  
Design Parameters ◽  
Digital Pulse ◽  
Neuromorphic System ◽  
The Impact

Spiking neural networks (SNN) offer a power efficient, biologically plausible learning paradigm by encoding information into spikes. The discovery of the memristor has accelerated the progress of spiking neuromorphic systems, as the intrinsic plasticity of the device makes it an ideal candidate to mimic a biological synapse. Despite providing a nanoscale form factor, non-volatility, and low-power operation, memristors suffer from device-level non-idealities, which impact system-level performance. To address these issues, this article presents a memristive crossbar-based neuromorphic system using unsupervised learning with twin-memristor synapses, fully digital pulse width modulated spike-timing-dependent plasticity, and homeostasis neurons. The implemented single-layer SNN was applied to a pattern-recognition task of classifying handwritten-digits. The performance of the system was analyzed by varying design parameters such as number of training epochs, neurons, and capacitors. Furthermore, the impact of memristor device non-idealities, such as device-switching mismatch, aging, failure, and process variations, were investigated and the resilience of the proposed system was demonstrated.

Betalogger: Smartphone Sensor-based Side-channel Attack Detection and Text Inference Using Language Modeling and Dense MultiLayer Neural Network

2021 ◽  
Vol 20 (5) ◽  
pp. 1-17
Author(s):  
Abdul Rehman Javed ◽  
Saif Ur Rehman ◽  
Mohib Ullah Khan ◽  
Mamoun Alazab ◽  
Habib Ullah Khan
Keyword(s):  
Neural Network ◽  
Language Modeling ◽  
Attack Detection ◽  
Machine Learning Algorithms ◽  
Second Phase ◽  
Sensitive Data ◽  
Phase Sequence ◽  
Smartphone Technology ◽  
Conventional Machine ◽  
Sequence Generator

With the recent advancement of smartphone technology in the past few years, smartphone usage has increased on a tremendous scale due to its portability and ability to perform many daily life tasks. As a result, smartphones have become one of the most valuable targets for hackers to perform cyberattacks, since the smartphone can contain individuals’ sensitive data. Smartphones are embedded with highly accurate sensors. This article proposes BetaLogger , an Android-based application that highlights the issue of leaking smartphone users’ privacy using smartphone hardware sensors (accelerometer, magnetometer, and gyroscope). BetaLogger efficiently infers the typed text (long or short) on a smartphone keyboard using Language Modeling and a Dense Multi-layer Neural Network (DMNN). BetaLogger is composed of two major phases: In the first phase, Text Inference Vector is given as input to the DMNN model to predict the target labels comprising the alphabet, and in the second phase, sequence generator module generate the output sequence in the shape of a continuous sentence. The outcomes demonstrate that BetaLogger generates highly accurate short and long sentences, and it effectively enhances the inference rate in comparison with conventional machine learning algorithms and state-of-the-art studies.

Internet financial supervision based on machine learning and improved neural network

2020 ◽  
pp. 1-12
Author(s):  
Cao Yanli
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Credit Risk ◽  
Machine Learning Algorithms ◽  
Financial Supervision ◽  
Loan Pricing ◽  
Comprehensive Management ◽  
Risk Pricing ◽  
Basel Agreement ◽  
Relevant Factors

The research on the risk pricing of Internet finance online loans not only enriches the theory and methods of online loan pricing, but also helps to improve the level of online loan risk pricing. In order to improve the efficiency of Internet financial supervision, this article builds an Internet financial supervision system based on machine learning algorithms and improved neural network algorithms. Moreover, on the basis of factor analysis and discretization of loan data, this paper selects the relatively mature Logistic regression model to evaluate the credit risk of the borrower and considers the comprehensive management of credit risk and the matching with income. In addition, according to the relevant provisions of the New Basel Agreement on expected losses and economic capital, starting from the relevant factors, this article combines the credit risk assessment results to obtain relevant factors through regional research and conduct empirical analysis. The research results show that the model constructed in this paper has certain reliability.

scholarly journals Biological modelling of a computational spiking neural network with neuronal avalanches

2017 ◽  
Vol 375 (2096) ◽  
pp. 20160286 ◽  
Author(s):  
Xiumin Li ◽  
Qing Chen ◽  
Fangzheng Xue
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Critical State ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Mathematical Methods ◽  
Computational Accuracy ◽  
Active Neuron ◽  
Computational Performance ◽  
Neuronal Avalanches

In recent years, an increasing number of studies have demonstrated that networks in the brain can self-organize into a critical state where dynamics exhibit a mixture of ordered and disordered patterns. This critical branching phenomenon is termed neuronal avalanches. It has been hypothesized that the homeostatic level balanced between stability and plasticity of this critical state may be the optimal state for performing diverse neural computational tasks. However, the critical region for high performance is narrow and sensitive for spiking neural networks (SNNs). In this paper, we investigated the role of the critical state in neural computations based on liquid-state machines, a biologically plausible computational neural network model for real-time computing. The computational performance of an SNN when operating at the critical state and, in particular, with spike-timing-dependent plasticity for updating synaptic weights is investigated. The network is found to show the best computational performance when it is subjected to critical dynamic states. Moreover, the active-neuron-dominant structure refined from synaptic learning can remarkably enhance the robustness of the critical state and further improve computational accuracy. These results may have important implications in the modelling of spiking neural networks with optimal computational performance. This article is part of the themed issue ‘Mathematical methods in medicine: neuroscience, cardiology and pathology’.

Sign in / Sign up

Export Citation Format

Share Document