Reservoir Computing for Scalable Hardware with Block‐Based Neural Network

2021 ◽  
Author(s):  
Kundo Lee ◽  
Tomoki Hamagami
Keyword(s):  
Neural Network ◽  
Reservoir Computing ◽  
Block Based ◽  
Scalable Hardware

scholarly journals Block-Based Compression and Corresponding Hardware Circuits for Sparse Activations

Sensors ◽  
2021 ◽  
Vol 21 (22) ◽  
pp. 7468
Author(s):  
Yui-Kai Weng ◽  
Shih-Hsu Huang ◽  
Hsu-Yu Kao
Keyword(s):  
Neural Network ◽  
Power Consumption ◽  
Convolutional Neural Network ◽  
Feature Maps ◽  
Benchmark Data ◽  
Data Volume ◽  
Block Based

In a CNN (convolutional neural network) accelerator, to reduce memory traffic and power consumption, there is a need to exploit the sparsity of activation values. Therefore, some research efforts have been paid to skip ineffectual computations (i.e., multiplications by zero). Different from previous works, in this paper, we point out the similarity of activation values: (1) in the same layer of a CNN model, most feature maps are either highly dense or highly sparse; (2) in the same layer of a CNN model, feature maps in different channels are often similar. Based on the two observations, we propose a block-based compression approach, which utilizes both the sparsity and the similarity of activation values to further reduce the data volume. Moreover, we also design an encoder, a decoder and an indexing module to support the proposed approach. The encoder is used to translate output activations into the proposed block-based compression format, while both the decoder and the indexing module are used to align nonzero values for effectual computations. Compared with previous works, benchmark data consistently show that the proposed approach can greatly reduce both memory traffic and power consumption.

scholarly journals Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

2020 ◽  
Vol 27 (3) ◽  
pp. 373-389 ◽  
Author(s):  
Ashesh Chattopadhyay ◽  
Pedram Hassanzadeh ◽  
Devika Subramanian
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Short Term Memory ◽  
Data Driven ◽  
Reservoir Computing ◽  
Short Term ◽  
Term Memory ◽  
Machine Learning Methods ◽  
Long Short Term Memory ◽  
Lorenz 96

Abstract. In this paper, the performance of three machine-learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multiscale spatiotemporal Lorenz 96 system is examined. The methods are an echo state network (ESN, which is a type of reservoir computing; hereafter RC–ESN), a deep feed-forward artificial neural network (ANN), and a recurrent neural network (RNN) with long short-term memory (LSTM; hereafter RNN–LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale (X), intermediate (Y), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC–ESN substantially outperforms ANN and RNN–LSTM for short-term predictions, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps equivalent to several Lyapunov timescales. The RNN–LSTM outperforms ANN, and both methods show some prediction skills too. Furthermore, even after losing the trajectory, data predicted by RC–ESN and RNN–LSTM have probability density functions (pdf's) that closely match the true pdf – even at the tails. The pdf of the data predicted using ANN, however, deviates from the true pdf. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems, such as weather and climate, are discussed.

scholarly journals Application of Artificial Neural Network for Image Noise Level Estimation in the SVD domain

Electronics ◽  
2019 ◽  
Vol 8 (2) ◽  
pp. 163 ◽  
Author(s):  
Emir Turajlic ◽  
Alen Begović ◽  
Namir Škaljo
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Noise Level ◽  
Low Noise ◽  
Singular Value ◽  
Fine Tuning ◽  
Noise Levels ◽  
Noise Level Estimation ◽  
Artificial Neural ◽  
Block Based

The blind additive white Gaussian noise level estimation is an important and a challenging area of digital image processing with numerous applications including image denoising and image segmentation. In this paper, a novel block-based noise level estimation algorithm is proposed. The algorithm relies on the artificial neural network to perform a complex image patch analysis in the singular value decomposition (SVD) domain and to evaluate noise level estimates. The algorithm exhibits the capacity to adjust the effective singular value tail length with respect to the observed noise levels. The results of comparative analysis show that the proposed ANN-based algorithm outperforms the alternative single stage block-based noise level estimating algorithm in the SVD domain in terms of mean square error (MSE) and average error for all considered choices of block size. The most significant improvements in MSE levels are obtained at low noise levels. For some test images, such as “Car” and “Girlface”, at σ = 1 , these improvements can be as high as 99% and 98.5%, respectively. In addition, the proposed algorithm eliminates the error-prone manual parameter fine-tuning and automates the entire noise level estimation process.

scholarly journals Active Shape Discrimination with Compliant Bodies as Reservoir Computers

2016 ◽  
Vol 22 (2) ◽  
pp. 241-268 ◽  
Author(s):  
Chris Johnson ◽  
Andrew Philippides ◽  
Philip Husbands
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Continuous Time ◽  
Complex Dynamics ◽  
The Body ◽  
Control Problems ◽  
Reservoir Computing ◽  
Shape Discrimination ◽  
Reflexive Behavior ◽  
Processing Information

Compliant bodies with complex dynamics can be used both to simplify control problems and to lead to adaptive reflexive behavior when engaged with the environment in the sensorimotor loop. By revisiting an experiment introduced by Beer and replacing the continuous-time recurrent neural network therein with reservoir computing networks abstracted from compliant bodies, we demonstrate that adaptive behavior can be produced by an agent in which the body is the main computational locus. We show that bodies with complex dynamics are capable of integrating, storing, and processing information in meaningful and useful ways, and furthermore that with the addition of the simplest of nervous systems such bodies can generate behavior that could equally be described as reflexive or minimally cognitive.

Sign in / Sign up

Export Citation Format

Share Document