scholarly journals Exploring the Impact of Variability in Resistance Distributions of RRAM on the Prediction Accuracy of Deep Learning Neural Networks

Electronics ◽  
2020 ◽  
Vol 9 (3) ◽  
pp. 414 ◽  
Author(s):  
Nagaraj Prabhu ◽  
Desmond Loy Jia Jun ◽  
Putu Dananjaya ◽  
Wen Lew ◽  
Eng Toh ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Prediction Accuracy ◽  
Random Access ◽  
Resistive Random Access Memory ◽  
Access Memory ◽  
Device Resistance ◽  
Measured Resistance ◽  
Deep Learning Neural Network ◽  
The Impact

In this work, we explore the use of the resistive random access memory (RRAM) device as a synapse for mimicking the trained weights linking neurons in a deep learning neural network (DNN) (AlexNet). The RRAM devices were fabricated in-house and subjected to 1000 bipolar read-write cycles to measure the resistances recorded for Logic-0 and Logic-1 (we demonstrate the feasibility of achieving eight discrete resistance states in the same device depending on the RESET stop voltage). DNN simulations have been performed to compare the relative error between the output of AlexNet Layer 1 (Convolution) implemented with the standard backpropagation (BP) algorithm trained weights versus the weights that are encoded using the measured resistance distributions from RRAM. The IMAGENET dataset is used for classification purpose here. We focus only on the Layer 1 weights in the AlexNet framework with 11 × 11 × 96 filters values coded into a binary floating point and substituted with the RRAM resistance values corresponding to Logic-0 and Logic-1. The impact of variability in the resistance states of RRAM for the low and high resistance states on the accuracy of image classification is studied by formulating a look-up table (LUT) for the RRAM (from measured I-V data) and comparing the convolution computation output of AlexNet Layer 1 with the standard outputs from the BP-based pre-trained weights. This is one of the first studies dedicated to exploring the impact of RRAM device resistance variability on the prediction accuracy of a convolutional neural network (CNN) on an AlexNet platform through a framework that requires limited actual device switching test data.

scholarly journals Three-Dimensional (3D) Vertical Resistive Random-Access Memory (VRRAM) Synapses for Neural Network Systems

Materials ◽  
2019 ◽  
Vol 12 (20) ◽  
pp. 3451 ◽  
Author(s):  
Wookyung Sun ◽  
Sujin Choi ◽  
Bokyung Kim ◽  
Junhee Park
Keyword(s):  
Neural Network ◽  
Structural Characteristics ◽  
3D Structure ◽  
Random Access ◽  
Random Access Memory ◽  
Resistive Random Access Memory ◽  
Operating Principle ◽  
Access Memory ◽  
Network Systems ◽  
Crossbar Array

Memristor devices are generally suitable for incorporation in neuromorphic systems as synapses because they can be integrated into crossbar array circuits with high area efficiency. In the case of a two-dimensional (2D) crossbar array, however, the size of the array is proportional to the neural network’s depth and the number of its input and output nodes. This means that a 2D crossbar array is not suitable for a deep neural network. On the other hand, synapses that use a memristor with a 3D structure are suitable for implementing a neuromorphic chip for a multi-layered neural network. In this study, we propose a new optimization method for machine learning weight changes that considers the structural characteristics of a 3D vertical resistive random-access memory (VRRAM) structure for the first time. The newly proposed synapse operating principle of the 3D VRRAM structure can simplify the complexity of a neuron circuit. This study investigates the operating principle of 3D VRRAM synapses with comb-shaped word lines and demonstrates that the proposed 3D VRRAM structure will be a promising solution for a high-density neural network hardware system.

scholarly journals A TaOx-Based RRAM with Improved Uniformity and Excellent Analog Characteristics by Local Dopant Engineering

Electronics ◽  
2021 ◽  
Vol 10 (20) ◽  
pp. 2451
Author(s):  
Yabo Qin ◽  
Zongwei Wang ◽  
Yaotian Ling ◽  
Yimao Cai ◽  
Ru Huang
Keyword(s):  
Neural Network ◽  
Formation Energy ◽  
Random Access ◽  
Random Access Memory ◽  
Resistive Random Access Memory ◽  
Collective Effects ◽  
Operating Voltage ◽  
Access Memory ◽  
Intelligent Algorithms ◽  
Network Mapping

Resistive random-access memory (RRAM) with the ability to store and process information has been considered to be one of the most promising emerging devices to emulate synaptic behavior and accelerate the computation of intelligent algorithms. However, variation and limited resistance levels impede RRAM as a synapse for weight storage in neural network mapping. In this work, we investigate a TaOx-based RRAM with Al ion local doping. Compared with a device without doping, the device with locally doped Al ion exhibits excellent uniformity and analog characteristics. The operating voltage and resistance states show tighter distributions. Over 150 adjustable resistance states can be achieved through tuning compliance current (CC) and reset stop voltage. Moreover, incremental resistance switching is available under optimized identical pulses. The improved uniformity and analog characteristics can be attributed to the collective effects of reduced oxygen vacancy (Vo) formation energy and weak conductive filaments induced by the local Al ion dopants.

Multilevel Cell Storage and Resistance Variability in Resistive Random Access Memory

2016 ◽  
Vol 1 (6) ◽  
Author(s):  
Amit Prakash ◽  
Hyunsang Hwang
Keyword(s):  
Low Cost ◽  
Random Access ◽  
Random Access Memory ◽  
Resistive Random Access Memory ◽  
Comprehensive Analysis ◽  
Multiple Resistance ◽  
Access Memory ◽  
Physical Mechanisms ◽  
Reliability Characteristics ◽  
The Impact

Abstract Multilevel per cell (MLC) storage in resistive random access memory (ReRAM) is attractive in achieving high-density and low-cost memory and will be required in future. In this chapter, MLC storage and resistance variability and reliability of multilevel in ReRAM are discussed. Different MLC operation schemes with their physical mechanisms and a comprehensive analysis of resistance variability have been provided. Various factors that can induce variability and their effect on the resistance margin between the multiple resistance levels are assessed. The reliability characteristics and the impact on MLC storage have also been assessed.

scholarly journals Analysis of Leakage Current of HfO2/TaOx-Based 3-D Vertical Resistive Random Access Memory Array

Micromachines ◽  
2021 ◽  
Vol 12 (6) ◽  
pp. 614
Author(s):  
Zhisheng Chen ◽  
Renjun Song ◽  
Qiang Huo ◽  
Qirui Ren ◽  
Chenrui Zhang ◽  
...  
Keyword(s):  
Leakage Current ◽  
Current Model ◽  
Random Access ◽  
Memory Cell ◽  
Random Access Memory ◽  
Resistive Random Access Memory ◽  
Memory Storage ◽  
Access Memory ◽  
Leakage Currents ◽  
The Impact

Three-dimensional vertical resistive random access memory (VRRAM) is proposed as a promising candidate for increasing resistive memory storage density, but the performance evaluation mechanism of 3-D VRRAM arrays is still not mature enough. The previous approach to evaluating the performance of 3-D VRRAM was based on the write and read margin. However, the leakage current (LC) of the 3-D VRRAM array is a concern as well. Excess leakage currents not only reduce the read/write tolerance and liability of the memory cell but also increase the power consumption of the entire array. In this article, a 3-D circuit HSPICE simulation is used to analyze the impact of the array size and operation voltage on the leakage current in the 3-D VRRAM architecture. The simulation results show that rapidly increasing leakage currents significantly affect the size of 3-D layers. A high read voltage is profitable for enhancing the read margin. However, the leakage current also increases. Alleviating this conflict requires a trade-off when setting the input voltage. A method to improve the array read/write efficiency is proposed by analyzing the influence of the multi-bit operations on the overall leakage current. Finally, this paper explores different methods to reduce the leakage current in the 3-D VRRAM array. The leakage current model proposed in this paper provides an efficient performance prediction solution for the initial design of 3-D VRRAM arrays.

scholarly journals Multi-resBind: a residual network-based multi-label classifier for in vivo RNA binding prediction and preference visualization

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Shitao Zhao ◽  
Michiaki Hamada
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Binding Sites ◽  
Prediction Accuracy ◽  
Large Scale ◽  
Rna Binding ◽  
Binding Prediction ◽  
Binding Preference ◽  
The Impact

Abstract Background Protein-RNA interactions play key roles in many processes regulating gene expression. To understand the underlying binding preference, ultraviolet cross-linking and immunoprecipitation (CLIP)-based methods have been used to identify the binding sites for hundreds of RNA-binding proteins (RBPs) in vivo. Using these large-scale experimental data to infer RNA binding preference and predict missing binding sites has become a great challenge. Some existing deep-learning models have demonstrated high prediction accuracy for individual RBPs. However, it remains difficult to avoid significant bias due to the experimental protocol. The DeepRiPe method was recently developed to solve this problem via introducing multi-task or multi-label learning into this field. However, this method has not reached an ideal level of prediction power due to the weak neural network architecture. Results Compared to the DeepRiPe approach, our Multi-resBind method demonstrated substantial improvements using the same large-scale PAR-CLIP dataset with respect to an increase in the area under the receiver operating characteristic curve and average precision. We conducted extensive experiments to evaluate the impact of various types of input data on the final prediction accuracy. The same approach was used to evaluate the effect of loss functions. Finally, a modified integrated gradient was employed to generate attribution maps. The patterns disentangled from relative contributions according to context offer biological insights into the underlying mechanism of protein-RNA interactions. Conclusions Here, we propose Multi-resBind as a new multi-label deep-learning approach to infer protein-RNA binding preferences and predict novel interactions. The results clearly demonstrate that Multi-resBind is a promising tool to predict unknown binding sites in vivo and gain biology insights into why the neural network makes a given prediction.

Sign in / Sign up

Export Citation Format

Share Document