scholarly journals UNIQ

2021 ◽  
Vol 37 (1--4) ◽  
pp. 1-15
Author(s):  
Chaim Baskin ◽  
Natan Liss ◽  
Eli Schwartz ◽  
Evgenii Zheltonozhskii ◽  
Raja Giryes ◽  
...  
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Low Complexity ◽  
High Accuracy ◽  
Trade Off ◽  
Training Time ◽  
Uniform Quantization ◽  
Novel Method ◽  
A Minor ◽  
Prior State

We present a novel method for neural network quantization. Our method, named UNIQ , emulates a non-uniform k -quantile quantizer and adapts the model to perform well with quantized weights by injecting noise to the weights at training time. As a by-product of injecting noise to weights, we find that activations can also be quantized to as low as 8-bit with only a minor accuracy degradation. Our non-uniform quantization approach provides a novel alternative to the existing uniform quantization techniques for neural networks. We further propose a novel complexity metric of number of bit operations performed (BOPs), and we show that this metric has a linear relation with logic utilization and power. We suggest evaluating the trade-off of accuracy vs. complexity (BOPs). The proposed method, when evaluated on ResNet18/34/50 and MobileNet on ImageNet, outperforms the prior state of the art both in the low-complexity regime and the high accuracy regime. We demonstrate the practical applicability of this approach, by implementing our non-uniformly quantized CNN on FPGA.

scholarly journals A design methodology for approximate multipliers in convolutional neural networks: A case of MNIST

2021 ◽  
Vol 10 (1) ◽  
pp. 1
Author(s):  
Kenta Shirane ◽  
Takahiro Yamamoto ◽  
Hiroyuki Tomiyama
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Design Methodology ◽  
Critical Path ◽  
High Accuracy ◽  
Path Delay ◽  
Trade Off ◽  
Critical Path Delay

In this paper, we present a case study on approximate multipliers for MNIST Convolutional Neural Network (CNN). We apply approximate multipliers with different bit-width to the convolution layer in MNIST CNN, evaluate the accuracy of MNIST classification, and analyze the trade-off between approximate multiplier’s area, critical path delay and the accuracy. Based on the results of the evaluation and analysis, we propose a design methodology for approximate multipliers. The approximate multipliers consist of some partial products, which are carefully selected according to the CNN input. With this methodology, we further reduce the area and the delay of the multipliers with keeping high accuracy of the MNIST classification.

scholarly journals Edge-Nodes Representation Neural Machine for Link Prediction

Algorithms ◽  
2019 ◽  
Vol 12 (1) ◽  
pp. 12 ◽  
Author(s):  
Guangluan Xu ◽  
Xiaoke Wang ◽  
Yang Wang ◽  
Daoyu Lin ◽  
Xian Sun ◽  
...  
Keyword(s):  
Neural Network ◽  
Formation Mechanism ◽  
Link Prediction ◽  
State Of The Art ◽  
Prediction Methods ◽  
Topological Features ◽  
Novel Method ◽  
Fully Connected ◽  
Two Sides

Link prediction is a task predicting whether there is a link between two nodes in a network. Traditional link prediction methods that assume handcrafted features (such as common neighbors) as the link’s formation mechanism are not universal. Other popular methods tend to learn the link’s representation, but they cannot represent the link fully. In this paper, we propose Edge-Nodes Representation Neural Machine (ENRNM), a novel method which can learn abundant topological features from the network as the link’s representation to promote the formation of the link. The ENRNM learns the link’s formation mechanism by combining the representation of edge and the representations of nodes on the two sides of the edge as link’s full representation. To predict the link’s existence, we train a fully connected neural network which can learn meaningful and abundant patterns. We prove that the features of edge and two nodes have the same importance in link’s formation. Comprehensive experiments are conducted on eight networks, experiment results demonstrate that the method ENRNM not only exceeds plenty of state-of-the-art link prediction methods but also performs very well on diverse networks with different structures and characteristics.

Convolutional Neural Network Considering the Effects of Noise for Bearing Fault Diagnosis

2020 ◽  
Author(s):  
Ilyoung Han ◽  
Jangbom Chai ◽  
Chanwoo Lim ◽  
Taeyun Kim
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Convolutional Neural Network ◽  
High Accuracy ◽  
Training Data ◽  
Bearing Fault ◽  
Bearing Fault Diagnosis ◽  
System Noise ◽  
Novel Method ◽  
Maintenance Activities

Abstract Convolutional Neural Network (CNN) is, in general, good at finding principal components of data. However, the characteristic components of the signals could often be obscured by system noise. Therefore, even though the CNN model is well-trained and predict with high accuracy, it may detect only the primary patterns of data which could be formed by system noise. They are, in fact, highly vulnerable to maintenance activities such as reassembly. In other words, CNN models could misdiagnose even with excellent performances. In this study, a novel method that combines the classification using CNN with the data preprocessing is proposed for bearing fault diagnosis. The proposed method is demonstrated by the following steps. First, training data is preprocessed so that the noise and the fault signature of the bearings are separated. Then, CNN models are developed and trained to learn significant features containing information of defects. Lastly, the CNN models are examined and validated whether they learn and extract the meaningful features or not.

scholarly journals Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories

2021 ◽  
Vol 9 ◽  
pp. 243-260
Author(s):  
Jakob Prange ◽  
Nathan Schneider ◽  
Vivek Srikumar
Keyword(s):  
Internal Structure ◽  
State Of The Art ◽  
High Accuracy ◽  
Structured Prediction ◽  
Long Tail ◽  
Constructive Models ◽  
Sizeable Fraction ◽  
The Many ◽  
Syntactic Derivation ◽  
Prior State

Abstract Although current CCG supertaggers achieve high accuracy on the standard WSJ test set, few systems make use of the categories’ internal structure that will drive the syntactic derivation during parsing. The tagset is traditionally truncated, discarding the many rare and complex category types in the long tail. However, supertags are themselves trees. Rather than give up on rare tags, we investigate constructive models that account for their internal structure, including novel methods for tree-structured prediction. Our best tagger is capable of recovering a sizeable fraction of the long-tail supertags and even generates CCG categories that have never been seen in training, while approximating the prior state of the art in overall tag accuracy with fewer parameters. We further investigate how well different approaches generalize to out-of-domain evaluation sets.

scholarly journals Robust Mouse Tracking in Complex Environments using Neural Networks

2018 ◽  
Author(s):  
Brian Q. Geuther ◽  
Sean P. Deats ◽  
Kai J. Fox ◽  
Steve A. Murray ◽  
Robert E. Braun ◽  
...  
Keyword(s):  
Neural Network ◽  
Environmental Conditions ◽  
State Of The Art ◽  
High Accuracy ◽  
General Purpose ◽  
Training Data ◽  
Network Architectures ◽  
Complex Environments ◽  
Behavioral Experiments ◽  
Modern Machine

AbstractThe ability to track animals accurately is critical for behavioral experiments. For video-based assays, this is often accomplished by manipulating environmental conditions to increase contrast between the animal and the background, in order to achieve proper foreground/background detection (segmentation). However, as behavioral paradigms become more sophisticated with ethologically relevant environments, the approach of modifying environmental conditions offers diminishing returns, particularly for scalable experiments. Currently, there is a need for methods to monitor behaviors over long periods of time, under dynamic environmental conditions, and in animals that are genetically and behaviorally heterogeneous. To address this need, we developed a state-of-the-art neural network-based tracker for mice, using modern machine vision techniques. We test three different neural network architectures to determine their performance on genetically diverse mice under varying environmental conditions. We find that an encoder-decoder segmentation neural network achieves high accuracy and speed with minimal training data. Furthermore, we provide a labeling interface, labeled training data, tuned hyperparameters, and a pre-trained network for the mouse behavior and neuroscience communities. This general-purpose neural network tracker can be easily extended to other experimental paradigms and even to other animals, through transfer learning, thus providing a robust, generalizable solution for biobehavioral research.

scholarly journals Knowledge Transfer for Out-of-Knowledge-Base Entities : A Graph Neural Network Approach

2017 ◽  
Author(s):  
Takuo Hamaguchi ◽  
Hidekazu Oiwa ◽  
Masashi Shimbo ◽  
Yuji Matsumoto
Keyword(s):  
Neural Network ◽  
Knowledge Base ◽  
State Of The Art ◽  
Test Time ◽  
Network Approach ◽  
Missing Information ◽  
Neural Network Approach ◽  
Training Time ◽  
Proposed Model ◽  
Graph Neural Networks

Knowledge base completion (KBC) aims to predict missing information in a knowledge base. In this paper, we address the out-of-knowledge-base (OOKB) entity problem in KBC: how to answer queries concerning test entities not observed at training time. Existing embedding-based KBC models assume that all test entities are available at training time, making it unclear how to obtain embeddings for new entities without costly retraining. To solve the OOKB entity problem without retraining, we use graph neural networks (Graph-NNs) to compute the embeddings of OOKB entities, exploiting the limited auxiliary knowledge provided at test time. The experimental results show the effectiveness of our proposed model in the OOKB setting. Additionally, in the standard KBC setting in which OOKB entities are not involved, our model achieves state-of-the-art performance on the WordNet dataset.

scholarly journals Algorithm for Detecting Characteristic Points on a Three-Dimensional, Whole-Body Human Scan

2020 ◽  
Vol 10 (4) ◽  
pp. 1342
Author(s):  
Michał Koźbiał ◽  
Łukasz Markiewicz ◽  
Robert Sitnik
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Characteristic Point ◽  
Three Dimensional ◽  
Whole Body ◽  
Data Preparation ◽  
Surface Projection ◽  
Characteristic Points ◽  
Novel Method ◽  
One Machine

Anthropometric landmarks obtained from three-dimensional (3D) body scans are widely used in medicine, civil engineering, and virtual reality. For all those fields, an acquisition of certain and accurate landmark positions is crucial for obtaining satisfying results. Manual marking is time-consuming and is affected by the subjectivity of the human operator. Therefore, an automatic approach has become increasingly popular. This paper provides a short survey of different attempts for automatic landmark localization, from which one machine learning-based method was further analyzed and extended in the case of input data preparation for a convolutional neural network (CNN). A novel method of data processing is presented which utilize a mid-surface projection followed by further unwrapping. The article emphasizes its significance and the way it affects the outcome of a deep neural network. The workflow and the detailed description of algorithms used are included in this paper. To validate the method, it was compared with the orthogonal projection used for the state-of-the-art approach. Datasets consisting of 200 specimens, acquired using both methods, were used for convolutional neural networks training and 20 for validation. In this paper, we used YOLO v.3 architecture for detection and ResNet-152 for classification. For each approach, localizations of 22 normalized body landmarks for 10 male and 10 female subjects of different ages and various postures were obtained. To compare the accuracy of approaches, errors and their distribution were measured for each characteristic point. Experiments confirmed that the mid-surface projections resulted, on average, in a 14% accuracy improvement and up to 15% enhancement of resistance on errors related to scan imperfections.

scholarly journals Gated Graph Attention Network for Cancer Prediction

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 1938
Author(s):  
Linling Qiu ◽  
Han Li ◽  
Meihong Wang ◽  
Xiaoli Wang
Keyword(s):  
Neural Network ◽  
Prediction Accuracy ◽  
State Of The Art ◽  
Network Models ◽  
The State ◽  
Neural Network Models ◽  
Attention Network ◽  
Training Time ◽  
Cancer Prediction ◽  
Gating Mechanism

With its increasing incidence, cancer has become one of the main causes of worldwide mortality. In this work, we mainly propose a novel attention-based neural network model named Gated Graph ATtention network (GGAT) for cancer prediction, where a gating mechanism (GM) is introduced to work with the attention mechanism (AM), to break through the previous work’s limitation of 1-hop neighbourhood reasoning. In this way, our GGAT is capable of fully mining the potential correlation between related samples, helping for improving the cancer prediction accuracy. Additionally, to simplify the datasets, we propose a hybrid feature selection algorithm to strictly select gene features, which significantly reduces training time without affecting prediction accuracy. To the best of our knowledge, our proposed GGAT achieves the state-of-the-art results in cancer prediction task on LIHC, LUAD, KIRC compared to other traditional machine learning methods and neural network models, and improves the accuracy by 1% to 2% on Cora dataset, compared to the state-of-the-art graph neural network methods.

scholarly journals A protection method of trained CNN model with a secret key from unauthorized access

2021 ◽  
Vol 10 ◽  
Author(s):  
AprilPyone Maungmaung ◽  
Hitoshi Kiya
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Network Models ◽  
The State ◽  
Secret Key ◽  
Neural Network Models ◽  
Unauthorized Access ◽  
Protection Method ◽  
Novel Method

In this paper, we propose a novel method for protecting convolutional neural network models with a secret key set so that unauthorized users without the correct key set cannot access trained models. The method enables us to protect not only from copyright infringement but also the functionality of a model from unauthorized access without any noticeable overhead. We introduce three block-wise transformations with a secret key set to generate learnable transformed images: pixel shuffling, negative/positive transformation, and format-preserving Feistel-based encryption. Protected models are trained by using transformed images. The results of experiments with the CIFAR and ImageNet datasets show that the performance of a protected model was close to that of non-protected models when the key set was correct, while the accuracy severely dropped when an incorrect key set was given. The protected model was also demonstrated to be robust against various attacks. Compared with the state-of-the-art model protection with passports, the proposed method does not have any additional layers in the network, and therefore, there is no overhead during training and inference processes.

scholarly journals Deep Transfer Learning for Vulnerable Road Users Detection using Smartphone Sensors Data

2020 ◽  
Vol 12 (21) ◽  
pp. 3508
Author(s):  
Mohammed Elhenawy ◽  
Huthaifa I. Ashqar ◽  
Mahmoud Masoud ◽  
Mohammed H. Almannaa ◽  
Andry Rakotonirainy ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Classification Accuracy ◽  
High Accuracy ◽  
Training Time ◽  
Road Users ◽  
Smartphone Sensors ◽  
Vulnerable Road Users ◽  
And Behavior

As the Autonomous Vehicle (AV) industry is rapidly advancing, the classification of non-motorized (vulnerable) road users (VRUs) becomes essential to ensure their safety and to smooth operation of road applications. The typical practice of non-motorized road users’ classification usually takes significant training time and ignores the temporal evolution and behavior of the signal. In this research effort, we attempt to detect VRUs with high accuracy be proposing a novel framework that includes using Deep Transfer Learning, which saves training time and cost, to classify images constructed from Recurrence Quantification Analysis (RQA) that reflect the temporal dynamics and behavior of the signal. Recurrence Plots (RPs) were constructed from low-power smartphone sensors without using GPS data. The resulted RPs were used as inputs for different pre-trained Convolutional Neural Network (CNN) classifiers including constructing 227 × 227 images to be used for AlexNet and SqueezeNet; and constructing 224 × 224 images to be used for VGG16 and VGG19. Results show that the classification accuracy of Convolutional Neural Network Transfer Learning (CNN-TL) reaches 98.70%, 98.62%, 98.71%, and 98.71% for AlexNet, SqueezeNet, VGG16, and VGG19, respectively. Moreover, we trained resnet101 and shufflenet for a very short time using one epoch of data and then used them as weak learners, which yielded 98.49% classification accuracy. The results of the proposed framework outperform other results in the literature (to the best of our knowledge) and show that using CNN-TL is promising for VRUs classification. Because of its relative straightforwardness, ability to be generalized and transferred, and potential high accuracy, we anticipate that this framework might be able to solve various problems related to signal classification.

Sign in / Sign up

Export Citation Format

Share Document