scholarly journals Neural Network for Low-Memory IoT Devices and MNIST Image Recognition Using Kernels Based on Logistic Map

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1432
Author(s):  
Andrei Velichko
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Classification Accuracy ◽  
Network Architecture ◽  
Chaotic Behavior ◽  
Logistic Map ◽  
Simple Algorithm ◽  
Research Perspective ◽  
Logistic Mapping ◽  
Iot Devices

This study presents a neural network which uses filters based on logistic mapping (LogNNet). LogNNet has a feedforward network structure, but possesses the properties of reservoir neural networks. The input weight matrix, set by a recurrent logistic mapping, forms the kernels that transform the input space to the higher-dimensional feature space. The most effective recognition of a handwritten digit from MNIST-10 occurs under chaotic behavior of the logistic map. The correlation of classification accuracy with the value of the Lyapunov exponent was obtained. An advantage of LogNNet implementation on IoT devices is the significant savings in memory used. At the same time, LogNNet has a simple algorithm and performance indicators comparable to those of the best resource-efficient algorithms available at the moment. The presented network architecture uses an array of weights with a total memory size from 1 to 29 kB and achieves a classification accuracy of 80.3–96.3%. Memory is saved due to the processor, which sequentially calculates the required weight coefficients during the network operation using the analytical equation of the logistic mapping. The proposed neural network can be used in implementations of artificial intelligence based on constrained devices with limited memory, which are integral blocks for creating ambient intelligence in modern IoT environments. From a research perspective, LogNNet can contribute to the understanding of the fundamental issues of the influence of chaos on the behavior of reservoir-type neural networks.

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

2020 ◽  
Vol 2020 (10) ◽  
pp. 54-62
Author(s):  
Oleksii VASYLIEV ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Statistical Data ◽  
Activation Function ◽  
Decision Making Process ◽  
Neural Network Architecture ◽  
Acceptable Accuracy ◽  
The Neural Network ◽  
Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

scholarly journals An efficient pruning scheme of deep neural networks for Internet of Things applications

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Chen Qi ◽  
Shibo Shen ◽  
Rongpeng Li ◽  
Zhifeng Zhao ◽  
Qing Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Internet Of Things ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Superior Performance ◽  
Compact Structure ◽  
Resource Limited ◽  
Benchmark Datasets ◽  
Iot Devices

AbstractNowadays, deep neural networks (DNNs) have been rapidly deployed to realize a number of functionalities like sensing, imaging, classification, recognition, etc. However, the computational-intensive requirement of DNNs makes it difficult to be applicable for resource-limited Internet of Things (IoT) devices. In this paper, we propose a novel pruning-based paradigm that aims to reduce the computational cost of DNNs, by uncovering a more compact structure and learning the effective weights therein, on the basis of not compromising the expressive capability of DNNs. In particular, our algorithm can achieve efficient end-to-end training that transfers a redundant neural network to a compact one with a specifically targeted compression rate directly. We comprehensively evaluate our approach on various representative benchmark datasets and compared with typical advanced convolutional neural network (CNN) architectures. The experimental results verify the superior performance and robust effectiveness of our scheme. For example, when pruning VGG on CIFAR-10, our proposed scheme is able to significantly reduce its FLOPs (floating-point operations) and number of parameters with a proportion of 76.2% and 94.1%, respectively, while still maintaining a satisfactory accuracy. To sum up, our scheme could facilitate the integration of DNNs into the common machine-learning-based IoT framework and establish distributed training of neural networks in both cloud and edge.

scholarly journals Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2852
Author(s):  
Parvathaneni Naga Srinivasu ◽  
Jalluri Gnana SivaSai ◽  
Muhammad Fazal Ijaz ◽  
Akash Kumar Bhoi ◽  
Wonjoon Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Skin Disease ◽  
Network Architecture ◽  
Large Scale ◽  
Short Term Memory ◽  
Convolutional Networks ◽  
Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

scholarly journals Reynolds averaged turbulence modelling using deep neural networks with embedded invariance

2016 ◽  
Vol 807 ◽  
pp. 155-166 ◽  
Author(s):  
Julia Ling ◽  
Andrew Kurzawski ◽  
Jeremy Templeton
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reynolds Stress ◽  
Network Architecture ◽  
Eddy Viscosity ◽  
Deep Neural Networks ◽  
Test Cases ◽  
Neural Network Architecture ◽  
Stress Anisotropy ◽  
Anisotropy Tensor

There exists significant demand for improved Reynolds-averaged Navier–Stokes (RANS) turbulence models that are informed by and can represent a richer set of turbulence physics. This paper presents a method of using deep neural networks to learn a model for the Reynolds stress anisotropy tensor from high-fidelity simulation data. A novel neural network architecture is proposed which uses a multiplicative layer with an invariant tensor basis to embed Galilean invariance into the predicted anisotropy tensor. It is demonstrated that this neural network architecture provides improved prediction accuracy compared with a generic neural network architecture that does not embed this invariance property. The Reynolds stress anisotropy predictions of this invariant neural network are propagated through to the velocity field for two test cases. For both test cases, significant improvement versus baseline RANS linear eddy viscosity and nonlinear eddy viscosity models is demonstrated.

Optimization Artificial Neural Network Using Artificial Bee Colony in Letter Recognition Classification

2020 ◽  
Vol 8 (4) ◽  
pp. 469
Author(s):  
I Gusti Ngurah Alit Indrawan ◽  
I Made Widiartha
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Classification Accuracy ◽  
Artificial Bee Colony Algorithm ◽  
Artificial Bee Colony ◽  
Letter Recognition ◽  
Bee Colony ◽  
Artificial Neural ◽  
Hidden Layer

Artificial Neural Networks or commonly abbreviated as ANN is one branch of science from the field of artificial intelligence which is often used to solve various problems in fields that involve grouping and pattern recognition. This research aims to classify Letter Recognition datasets using Artificial Neural Networks which are weighted optimally using the Artificial Bee Colony algorithm. The best classification accuracy results from this study were 92.85% using a combination of 4 hidden layers with each hidden layer containing 10 neurons.

Fourier Neural Networks and Generalized Single Hidden Layer Networks in Aircraft Engine Fault Diagnostics

2005 ◽  
Vol 128 (4) ◽  
pp. 773-782 ◽  
Author(s):  
H. S. Tan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Fourier Transforms ◽  
Aircraft Engine ◽  
Fault Classification ◽  
Noise Robustness ◽  
Fault Diagnostics ◽  
Multiple Faults ◽  
Hidden Layer

The conventional approach to neural network-based aircraft engine fault diagnostics has been mainly via multilayer feed-forward systems with sigmoidal hidden neurons trained by back propagation as well as radial basis function networks. In this paper, we explore two novel approaches to the fault-classification problem using (i) Fourier neural networks, which synthesizes the approximation capability of multidimensional Fourier transforms and gradient-descent learning, and (ii) a class of generalized single hidden layer networks (GSLN), which self-structures via Gram-Schmidt orthonormalization. Using a simulation program for the F404 engine, we generate steady-state engine parameters corresponding to a set of combined two-module deficiencies and require various neural networks to classify the multiple faults. We show that, compared to the conventional network architecture, the Fourier neural network exhibits stronger noise robustness and the GSLNs converge at a much superior speed.

scholarly journals Dynamic versus static neural network model for rainfall forecasting at Klang River Basin, Malaysia

2012 ◽  
Vol 16 (4) ◽  
pp. 1151-1169 ◽  
Author(s):  
A. El-Shafie ◽  
A. Noureldin ◽  
M. Taha ◽  
A. Hussain ◽  
M. Mukhlisin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Rainfall Time Series ◽  
Hydrological Process ◽  
Multi Layer Perceptron ◽  
Rainfall Forecasting ◽  
Unseen Data

Abstract. Rainfall is considered as one of the major components of the hydrological process; it takes significant part in evaluating drought and flooding events. Therefore, it is important to have an accurate model for rainfall forecasting. Recently, several data-driven modeling approaches have been investigated to perform such forecasting tasks as multi-layer perceptron neural networks (MLP-NN). In fact, the rainfall time series modeling involves an important temporal dimension. On the other hand, the classical MLP-NN is a static and has a memoryless network architecture that is effective for complex nonlinear static mapping. This research focuses on investigating the potential of introducing a neural network that could address the temporal relationships of the rainfall series. Two different static neural networks and one dynamic neural network, namely the multi-layer perceptron neural network (MLP-NN), radial basis function neural network (RBFNN) and input delay neural network (IDNN), respectively, have been examined in this study. Those models had been developed for the two time horizons for monthly and weekly rainfall forecasting at Klang River, Malaysia. Data collected over 12 yr (1997–2008) on a weekly basis and 22 yr (1987–2008) on a monthly basis were used to develop and examine the performance of the proposed models. Comprehensive comparison analyses were carried out to evaluate the performance of the proposed static and dynamic neural networks. Results showed that the MLP-NN neural network model is able to follow trends of the actual rainfall, however, not very accurately. RBFNN model achieved better accuracy than the MLP-NN model. Moreover, the forecasting accuracy of the IDNN model was better than that of static network during both training and testing stages, which proves a consistent level of accuracy with seen and unseen data.

scholarly journals Concatenated Convolutional Neural Network Model for Cuffless Blood Pressure Estimation Using Fuzzy Recurrence Properties of PPG Signals

2022 ◽  
Author(s):  
Ali Bahari Malayeri ◽  
Mohammad Bagher Khodabakhshi
Keyword(s):  
Neural Network ◽  
Blood Pressure ◽  
Neural Networks ◽  
Network Architecture ◽  
Spatial Information ◽  
Recurrence Plot ◽  
Estimation Model ◽  
British Hypertension Society ◽  
Blood Pressure Estimation ◽  
Cuffless Blood Pressure

Abstract Due to the importance of continuous monitoring of blood pressure (BP) in controlling hypertension, the topic of cuffless blood pressure (BP) estimation has been widely studied in recent years. A most important approach is to explore the nonlinear mapping between the recorded peripheral signals and the BP values which is usually conducted by deep neural networks. Because of the sequence-based pseudo periodic nature of peripheral signals such as photoplethysmogram (PPG), a proper estimation model needed to be equipped with the 1-dimensional (1-D) and recurrent layers. This, in turn, limits the usage of 2-dimensional (2-D) layers adopted in convolutional neural networks (CNN) for embedding spatial information in the model. In this study, considering the advantage of chaotic approaches, the recurrence characterization of peripheral signals was taken into account by a visual 2-D representation of PPG in phase space through fuzzy recurrence plot (FRP). FRP not only provides a beneficial framework for capturing the spatial properties of input signals but also creates a reliable approach for embedding the pseudo periodic properties to the neural models without using recurrent layers. Moreover, this study proposes a novel deep neural network architecture that combines the morphological features extracted simultaneously from two upgraded 1-D and 2-D CNNs capturing the temporal and spatial dependencies of PPGs in systolic and diastolic BP estimation. The model has been fed with the 1-D PPG sequences and the corresponding 2-D FRPs from two separate routes. The performance of the proposed framework was examined on the well-known public dataset, namely, Multi-Parameter Intelligent in Intensive Care II. Our scheme is analyzed and compared with the literature in terms of the requirements of the standards set by the British Hypertension Society (BHS) and the Association for the Advancement of Medical Instrumentation (AAMI). The proposed model met the AAMI requirements, and it achieved a grade of A as stated by the BHS standard. In addition, its mean absolute errors (MAE) and standard deviation for both systolic and diastolic blood pressure estimations were considerably low, 3.05±5.26 mmHg and 1.58±2.6 mmHg, in turn.

scholarly journals Towards Heterogeneous Multi-Agent Reinforcement Learning with Graph Neural Networks

2020 ◽  
Author(s):  
Douglas Meneghetti ◽  
Reinaldo Bianchi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Communication Channels ◽  
Neural Network Architecture ◽  
Graph Representations ◽  
Labeled Graph ◽  
Multiple Agent ◽  
Multi Agent ◽  
Graph Neural Networks

This work proposes a neural network architecture that learns policies for multiple agent classes in a heterogeneous multi-agent reinforcement setting. The proposed network uses directed labeled graph representations for states, encodes feature vectors of different sizes for different entity classes, uses relational graph convolution layers to model different communication channels between entity types and learns distinct policies for different agent classes, sharing parameters wherever possible. Results have shown that specializing the communication channels between entity classes is a promising step to achieve higher performance in environments composed of heterogeneous entities.

Sign in / Sign up

Export Citation Format

Share Document