Training fully connected networks with resistive memories: impact of device failures

2019 ◽  
Vol 213 ◽  
pp. 371-391 ◽  
Author(s):  
Louis P. Romero ◽  
Stefano Ambrogio ◽  
Massimo Giordano ◽  
Giorgio Cristiano ◽  
Martina Bodini ◽  
...  
Keyword(s):  
Test Accuracy ◽  
Fully Connected ◽  
The Impact ◽  
Fully Connected Networks

This paper explores the impact of device failures, NVM conductances that may contribute read current but which cannot be programmed, on DNN training and test accuracy.

scholarly journals Convergence Behavior of DNNs with Mutual-Information-Based Regularization

Entropy ◽  
2020 ◽  
Vol 22 (7) ◽  
pp. 727 ◽  
Author(s):  
Hlynur Jónsson ◽  
Giovanni Cherubini ◽  
Evangelos Eleftheriou
Keyword(s):  
Neural Networks ◽  
Mutual Information ◽  
Low Complexity ◽  
High Dimensional ◽  
Test Accuracy ◽  
Compression Phase ◽  
Hidden Layer ◽  
Low Dimensional ◽  
Fully Connected ◽  
Fully Connected Networks

Information theory concepts are leveraged with the goal of better understanding and improving Deep Neural Networks (DNNs). The information plane of neural networks describes the behavior during training of the mutual information at various depths between input/output and hidden-layer variables. Previous analysis revealed that most of the training epochs are spent on compressing the input, in some networks where finiteness of the mutual information can be established. However, the estimation of mutual information is nontrivial for high-dimensional continuous random variables. Therefore, the computation of the mutual information for DNNs and its visualization on the information plane mostly focused on low-complexity fully connected networks. In fact, even the existence of the compression phase in complex DNNs has been questioned and viewed as an open problem. In this paper, we present the convergence of mutual information on the information plane for a high-dimensional VGG-16 Convolutional Neural Network (CNN) by resorting to Mutual Information Neural Estimation (MINE), thus confirming and extending the results obtained with low-dimensional fully connected networks. Furthermore, we demonstrate the benefits of regularizing a network, especially for a large number of training epochs, by adopting mutual information estimates as additional terms in the loss function characteristic of the network. Experimental results show that the regularization stabilizes the test accuracy and significantly reduces its variance.

scholarly journals Classification of multiwavelength transients with Machine Learning

2020 ◽  
Author(s):  
K Sooknunan ◽  
M Lochner ◽  
Bruce A Bassett ◽  
H V Peiris ◽  
R Fender ◽  
...  
Keyword(s):  
Machine Learning ◽  
Small Sample ◽  
Light Curves ◽  
Machine Learning Techniques ◽  
Optical Data ◽  
Test Time ◽  
Test Accuracy ◽  
Training Set ◽  
The Impact

Abstract With the advent of powerful telescopes such as the Square Kilometer Array and the Vera C. Rubin Observatory, we are entering an era of multiwavelength transient astronomy that will lead to a dramatic increase in data volume. Machine learning techniques are well suited to address this data challenge and rapidly classify newly detected transients. We present a multiwavelength classification algorithm consisting of three steps: (1) interpolation and augmentation of the data using Gaussian processes; (2) feature extraction using wavelets; (3) classification with random forests. Augmentation provides improved performance at test time by balancing the classes and adding diversity into the training set. In the first application of machine learning to the classification of real radio transient data, we apply our technique to the Green Bank Interferometer and other radio light curves. We find we are able to accurately classify most of the eleven classes of radio variables and transients after just eight hours of observations, achieving an overall test accuracy of 78%. We fully investigate the impact of the small sample size of 82 publicly available light curves and use data augmentation techniques to mitigate the effect. We also show that on a significantly larger simulated representative training set that the algorithm achieves an overall accuracy of 97%, illustrating that the method is likely to provide excellent performance on future surveys. Finally, we demonstrate the effectiveness of simultaneous multiwavelength observations by showing how incorporating just one optical data point into the analysis improves the accuracy of the worst performing class by 19%.

scholarly journals Classification of Shoulder X-ray Images with Deep Learning Ensemble Models

2021 ◽  
Vol 11 (6) ◽  
pp. 2723
Author(s):  
Fatih Uysal ◽  
Fırat Hardalaç ◽  
Ozan Peker ◽  
Tolga Tolunay ◽  
Nil Tokgöz
Keyword(s):  
Deep Learning ◽  
Performance Test ◽  
The Body ◽  
Test Accuracy ◽  
Cohen’S Kappa ◽  
X Ray ◽  
Cohen's Kappa ◽  
Auc Value ◽  
Magnetic Resonance Imaging Mri ◽  
Fully Connected

Fractures occur in the shoulder area, which has a wider range of motion than other joints in the body, for various reasons. To diagnose these fractures, data gathered from X-radiation (X-ray), magnetic resonance imaging (MRI), or computed tomography (CT) are used. This study aims to help physicians by classifying shoulder images taken from X-ray devices as fracture/non-fracture with artificial intelligence. For this purpose, the performances of 26 deep learning-based pre-trained models in the detection of shoulder fractures were evaluated on the musculoskeletal radiographs (MURA) dataset, and two ensemble learning models (EL1 and EL2) were developed. The pre-trained models used are ResNet, ResNeXt, DenseNet, VGG, Inception, MobileNet, and their spinal fully connected (Spinal FC) versions. In the EL1 and EL2 models developed using pre-trained models with the best performance, test accuracy was 0.8455, 0.8472, Cohen’s kappa was 0.6907, 0.6942 and the area that was related with fracture class under the receiver operating characteristic (ROC) curve (AUC) was 0.8862, 0.8695. As a result of 28 different classifications in total, the highest test accuracy and Cohen’s kappa values were obtained in the EL2 model, and the highest AUC value was obtained in the EL1 model.

Clique Here: On the Distributed Complexity in Fully-Connected Networks

2016 ◽  
Vol 26 (01) ◽  
pp. 1650004 ◽  
Author(s):  
Benny Applebaum ◽  
Dariusz R. Kowalski ◽  
Boaz Patt-Shamir ◽  
Adi Rosén
Keyword(s):  
Boolean Function ◽  
Boolean Functions ◽  
Message Passing ◽  
Constant Number ◽  
Running Time ◽  
Message Size ◽  
Explicit Function ◽  
Randomized Protocols ◽  
Fully Connected ◽  
Fully Connected Networks

We consider a message passing model with n nodes, each connected to all other nodes by a link that can deliver a message of B bits in a time unit (typically, B = O(log n)). We assume that each node has an input of size L bits (typically, L = O(n log n)) and the nodes cooperate in order to compute some function (i.e., perform a distributed task). We are interested in the number of rounds required to compute the function. We give two results regarding this model. First, we show that most boolean functions require ‸ L/B ‹ − 1 rounds to compute deterministically, and that even if we consider randomized protocols that are allowed to err, the expected running time remains [Formula: see text] for most boolean function. Second, trying to find explicit functions that require superconstant time, we consider the pointer chasing problem. In this problem, each node i is given an array Ai of length n whose entries are in [n], and the task is to find, for any [Formula: see text], the value of [Formula: see text]. We give a deterministic O(log n/ log log n) round protocol for this function using message size B = O(log n), a slight but non-trivial improvement over the O(log n) bound provided by standard “pointer doubling.” The question of an explicit function (or functionality) that requires super constant number of rounds in this setting remains, however, open.

The impact of selection bias on measurement of noninvasive test accuracy

1993 ◽  
Vol 72 (2) ◽  
pp. 223-225 ◽  
Author(s):  
David M. Salerno ◽  
Kyuhyun Wang ◽  
Irvin F. Goldenberg ◽  
Robert A. Van Tassel
Keyword(s):  
Selection Bias ◽  
Test Accuracy ◽  
Noninvasive Test ◽  
The Impact

scholarly journals Binary and Multiclass Text Classification by Means of Separable Convolutional Neural Network

Inventions ◽  
2021 ◽  
Vol 6 (4) ◽  
pp. 70
Author(s):  
Elena Solovyeva ◽  
Ali Abdullah
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Recurrent Neural Networks ◽  
Low Cost ◽  
Computational Cost ◽  
High Accuracy ◽  
Activation Functions ◽  
Fully Connected ◽  
Fully Connected Networks

In this paper, the structure of a separable convolutional neural network that consists of an embedding layer, separable convolutional layers, convolutional layer and global average pooling is represented for binary and multiclass text classifications. The advantage of the proposed structure is the absence of multiple fully connected layers, which is used to increase the classification accuracy but raises the computational cost. The combination of low-cost separable convolutional layers and a convolutional layer is proposed to gain high accuracy and, simultaneously, to reduce the complexity of neural classifiers. Advantages are demonstrated at binary and multiclass classifications of written texts by means of the proposed networks under the sigmoid and Softmax activation functions in convolutional layer. At binary and multiclass classifications, the accuracy obtained by separable convolutional neural networks is higher in comparison with some investigated types of recurrent neural networks and fully connected networks.

scholarly journals Monitor-Watcher Use, Nurses’ Knowledge of Electrocardiographic Monitoring, and Arrhythmia Detection

2021 ◽  
Vol 30 (1) ◽  
pp. 38-44
Author(s):  
Marjorie Funk ◽  
Kristopher P. Fennie ◽  
Krista A. Knudson ◽  
Halley Ruppel
Keyword(s):  
Intensive Care Unit ◽  
Closed Loop ◽  
Test Accuracy ◽  
Ecg Monitoring ◽  
Knowledge Test ◽  
Arrhythmia Detection ◽  
Electrocardiographic Monitoring ◽  
Online Test ◽  
The Impact ◽  
Nurses Knowledge

Background Electrocardiographic telemetry monitors are ubiquitous in hospitals. Dedicated monitor watchers, either on the unit or in a centralized location, are often responsible for observing telemetry monitors and responding to their alarms. The impact of use of monitor watchers is not known. Objectives To evaluate the association of monitor-watcher use with (1) nurses’ knowledge of electrocardiographic (ECG) monitoring and (2) accuracy of arrhythmia detection. Methods Baseline data from 37 non–intensive care unit cardiac patient care areas in 17 hospitals in the Practical Use of the Latest Standards for Electrocardiography trial were analyzed. Nurses’ knowledge (n = 1136 nurses) was measured using a validated, 20-item online test. Accuracy of arrhythmia detection (n = 1189 patients) was assessed for 5 consecutive days by comparing arrhythmias stored in the monitor with nurses’ documentation. Multiple regression was used to evaluate the association of use of monitor watchers with scores on the ECG-monitoring knowledge test. The association of monitor-watcher use with accuracy of arrhythmia detection was examined by χ2 analysis. Results Of the 37 units, 13 (35%) had monitor watchers. Use of monitor watchers was not independently associated with ECG-monitoring knowledge (P = .08). The presence of monitor watchers also was not significantly associated with the accuracy of arrhythmia detection (P = .94). Conclusion Although the use of monitor watchers was not associated with diminished nurses’ knowledge of ECG monitoring, it also was not associated with more accurate arrhythmia detection. If implementing a monitor-watcher program, critical safety points, such as ensuring closed-loop communication, must be considered.

scholarly journals A Node-Based SIRS Epidemic Model with Infective Media on Complex Networks

Complexity ◽  
2019 ◽  
Vol 2019 ◽  
pp. 1-14 ◽  
Author(s):  
Leyi Zheng ◽  
Longkun Tang
Keyword(s):  
Epidemic Model ◽  
Small World ◽  
Spreading Rate ◽  
Node Degree ◽  
Strong Positive Correlation ◽  
Globally Asymptotically Stable ◽  
Sirs Epidemic Model ◽  
Theoretical Investigations ◽  
Fully Connected ◽  
The Impact

We focus on the node-based epidemic modeling for networks, introduce the propagation medium, and propose a node-based Susceptible-Infected-Recovered-Susceptible (SIRS) epidemic model with infective media. Theoretical investigations show that the endemic equilibrium is globally asymptotically stable. Numerical examples of three typical network structures also verify the theoretical results. Furthermore, comparison between network node degree and its infected percents implies that there is a strong positive correlation between both; namely, the node with bigger degree is infected with more percents. Finally, we discuss the impact of the epidemic spreading rate of media as well as the effective recovered rate on the network average infected state. Theoretical and numerical results show that (1) network average infected percents go up (down) with the increase of the infected rate of media (the effective recovered rate); (2) the infected rate of media has almost no influence on network average infected percents for the fully connected network and NW small-world network; (3) network average infected percents decrease exponentially with the increase of the effective recovered rate, implying that the percents can be controlled at low level by an appropriate large effective recovered rate.

scholarly journals Knocking and Listening: Learning Mechanical Impulse Response for Understanding Surface Characteristics

Sensors ◽  
2020 ◽  
Vol 20 (2) ◽  
pp. 369 ◽  
Author(s):  
Semin Ryu ◽  
Seung-Chan Kim
Keyword(s):  
Intelligent System ◽  
Surface Characteristics ◽  
Machine Learning Techniques ◽  
Test Accuracy ◽  
Learning Approaches ◽  
Well Test ◽  
Mechanical Responses ◽  
Learning Techniques ◽  
The Impact ◽  
Mechanical Impulse

Inspired by spiders that can generate and sense vibrations to obtain information regarding a substrate, we propose an intelligent system that can recognize the type of surface being touched by knocking the surface and listening to the vibrations. Hence, we developed a system that is equipped with an electromagnetic hammer for hitting the ground and an accelerometer for measuring the mechanical responses induced by the impact. We investigate the feasibility of sensing 10 different daily surfaces through various machine-learning techniques including recent deep-learning approaches. Although some test surfaces are similar, experimental results show that our system can recognize 10 different surfaces remarkably well (test accuracy of 98.66%). In addition, our results without directly hitting the surface (internal impact) exhibited considerably high test accuracy (97.51%). Finally, we conclude this paper with the limitations and future directions of the study.

Sign in / Sign up

Export Citation Format

Share Document