An efficient learning method for layered neural networks based on selection of training data and input characteristics of an output layer unit

2012 ◽  
Vol 95 (4) ◽  
pp. 57-67
Author(s):  
Isao Taguchi ◽  
Yasuo Sugai
Keyword(s):  
Neural Networks ◽  
Training Data ◽  
Learning Method ◽  
Output Layer ◽  
Efficient Learning ◽  
Selection Of

scholarly journals Patch-Based Change Detection Method for SAR Images with Label Updating Strategy

2021 ◽  
Vol 13 (7) ◽  
pp. 1236
Author(s):  
Yuanjun Shu ◽  
Wei Li ◽  
Menglong Yang ◽  
Peng Cheng ◽  
Songchen Han
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Change Detection ◽  
Detection Method ◽  
Training Data ◽  
Synthetic Aperture ◽  
Learning Method ◽  
Sar Images ◽  
Two Stage ◽  
Second Stage

Convolutional neural networks (CNNs) have been widely used in change detection of synthetic aperture radar (SAR) images and have been proven to have better precision than traditional methods. A two-stage patch-based deep learning method with a label updating strategy is proposed in this paper. The initial label and mask are generated at the pre-classification stage. Then a two-stage updating strategy is applied to gradually recover changed areas. At the first stage, diversity of training data is gradually restored. The output of the designed CNN network is further processed to generate a new label and a new mask for the following learning iteration. As the diversity of data is ensured after the first stage, pixels within uncertain areas can be easily classified at the second stage. Experiment results on several representative datasets show the effectiveness of our proposed method compared with several existing competitive methods.

scholarly journals Data Size Requirement for Forecasting Daily Crude Oil Price with Neural Networks

2019 ◽  
Vol 66 (3) ◽  
pp. 363-388
Author(s):  
Serkan Aras ◽  
Manel Hamdi
Keyword(s):  
Neural Networks ◽  
Crude Oil ◽  
Significant Interaction ◽  
Statistical Tests ◽  
Oil Price ◽  
Training Data ◽  
Forecasting Model ◽  
Crude Oil Price ◽  
Crude Oil Prices ◽  
Selection Of

When the literature regarding applications of neural networks is investigated, it appears that a substantial issue is what size the training data should be when modelling a time series through neural networks. The aim of this paper is to determine the size of training data to be used to construct a forecasting model via a multiple-breakpoint test and compare its performance with two general methods, namely, using all available data and using just two years of data. Furthermore, the importance of the selection of the final neural network model is investigated in detail. The results obtained from daily crude oil prices indicate that the data from the last structural change lead to simpler architectures of neural networks and have an advantage in reaching more accurate forecasts in terms of MAE value. In addition, the statistical tests show that there is a statistically significant interaction between data size and stopping rule.

Information Based Selection of Neural Networks Training Data for S.I. Engine Mapping

2001 ◽  
Author(s):  
Ivan Arsie ◽  
Fabrizio Marotta ◽  
Cesare Pianese ◽  
Gianfranco Rizzo
Keyword(s):  
Neural Networks ◽  
Training Data ◽  
Selection Of

scholarly journals Logic Tensor Networks for Semantic Image Interpretation

2017 ◽  
Author(s):  
Ivan Donadello ◽  
Luciano Serafini ◽  
Artur d'Avila Garcez
Keyword(s):  
Neural Networks ◽  
Image Interpretation ◽  
Background Knowledge ◽  
Learning System ◽  
Training Data ◽  
Reasoning Under Uncertainty ◽  
Tensor Networks ◽  
Logical Constraints ◽  
Bounding Boxes ◽  
Efficient Learning

Semantic Image Interpretation (SII) is the task of extracting structured semantic descriptions from images. It is widely agreed that the combined use of visual data and background knowledge is of great importance for SII. Recently, Statistical Relational Learning (SRL) approaches have been developed for reasoning under uncertainty and learning in the presence of data and rich knowledge. Logic Tensor Networks (LTNs) are a SRL framework which integrates neural networks with first-order fuzzy logic to allow (i) efficient learning from noisy data in the presence of logical constraints, and (ii) reasoning with logical formulas describing general properties of the data. In this paper, we develop and apply LTNs to two of the main tasks of SII, namely, the classification of an image's bounding boxes and the detection of the relevant part-of relations between objects. To the best of our knowledge, this is the first successful application of SRL to such SII tasks. The proposed approach is evaluated on a standard image processing benchmark. Experiments show that background knowledge in the form of logical constraints can improve the performance of purely data-driven approaches, including the state-of-the-art Fast Region-based Convolutional Neural Networks (Fast R-CNN). Moreover, we show that the use of logical background knowledge adds robustness to the learning system when errors are present in the labels of the training data.

scholarly journals Domain-Adversarial Based Model with Phonological Knowledge for Cross-Lingual Speech Recognition

Electronics ◽  
2021 ◽  
Vol 10 (24) ◽  
pp. 3172
Author(s):  
Qingran Zhan ◽  
Xiang Xie ◽  
Chenguang Hu ◽  
Juan Zuluaga-Gomez ◽  
Jing Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Speech Recognition ◽  
Training Data ◽  
Target Language ◽  
Learning Method ◽  
Acoustic Features ◽  
Adversarial Learning ◽  
Phonological Knowledge ◽  
Cross Lingual

Phonological-based features (articulatory features, AFs) describe the movements of the vocal organ which are shared across languages. This paper investigates a domain-adversarial neural network (DANN) to extract reliable AFs, and different multi-stream techniques are used for cross-lingual speech recognition. First, a novel universal phonological attributes definition is proposed for Mandarin, English, German and French. Then a DANN-based AFs detector is trained using source languages (English, German and French). When doing the cross-lingual speech recognition, the AFs detectors are used to transfer the phonological knowledge from source languages (English, German and French) to the target language (Mandarin). Two multi-stream approaches are introduced to fuse the acoustic features and cross-lingual AFs. In addition, the monolingual AFs system (i.e., the AFs are directly extracted from the target language) is also investigated. Experiments show that the performance of the AFs detector can be improved by using convolutional neural networks (CNN) with a domain-adversarial learning method. The multi-head attention (MHA) based multi-stream can reach the best performance compared to the baseline, cross-lingual adaptation approach, and other approaches. More specifically, the MHA-mode with cross-lingual AFs yields significant improvements over monolingual AFs with the restriction of training data size and, which can be easily extended to other low-resource languages.

Power Function Error Initialization Can Improve Convergence of Backpropagation Learning in Neural Networks for Classification

2021 ◽  
pp. 1-33
Author(s):  
Andreas Knoblauch
Keyword(s):  
Neural Networks ◽  
Training Data ◽  
Cross Entropy ◽  
Loss Functions ◽  
Power Functions ◽  
Output Layer ◽  
New Family ◽  
Target Values ◽  
The Difference ◽  
Backpropagation Learning

Abstract Abstract supervised learning corresponds to minimizing a loss or cost function expressing the differences between model predictions yn and the target values tn given by the training data. In neural networks, this means backpropagating error signals through the transposed weight matrixes from the output layer toward the input layer. For this, error signals in the output layer are typically initialized by the difference yn - tn, which is optimal for several commonly used loss functions like cross-entropy or sum of squared errors. Here I evaluate a more general error initialization method using power functions |yn - tn|q for q>0, corresponding to a new family of loss functions that generalize cross-entropy. Surprisingly, experiments on various learning tasks reveal that a proper choice of q can significantly improve the speed and convergence of backpropagation learning, in particular in deep and recurrent neural networks. The results suggest two main reasons for the observed improvements. First, compared to cross-entropy, the new loss functions provide better fits to the distribution of error signals in the output layer and therefore maximize the model's likelihood more efficiently. Second, the new error initialization procedure may often provide a better gradient-to-loss ratio over a broad range of neural output activity, thereby avoiding flat loss landscapes with vanishing gradients.

Results of Bias-variance Tests on Multi-layer Perceptron Neural Networks

2001 ◽  
Vol 5 (5) ◽  
pp. 300-305
Author(s):  
Wimpie D. Nortje ◽  
◽  
Johann E. W. Holm ◽  
Gerhard P. Hancke ◽  
Imre. J. Rudas ◽  
...  
Keyword(s):  
Neural Networks ◽  
Bayesian Networks ◽  
Training Data ◽  
Computational Time ◽  
Bayesian Techniques ◽  
Multiple Networks ◽  
The Neural Networks ◽  
Computational Errors ◽  
Bias Variance ◽  
Selection Of

Training neural networks involves selection of a set of network parameters, or weights, on account of fitting a non-linear model to data. Due to the bias in the training data and small computational errors, the neural networks’ opinions are biased. Some improvement is possible when multiple networks are used to do the classification. This approach is similar to taking the average of a number of biased opinions in order to remove some of the bias that resulted from training. Bayesian networks are effective in removing some of the bias associated with training, but Bayesian techniques are tedious in terms of computational time. It is for this reason that alternatives to Bayesian networks are investigated.

scholarly journals IMPLEMENTASI JARINGAN SYARAF TIRUAN BACKPROPAGATION DENGAN ALGORITMA CONJUGATE GRADIENT UNTUK KLASIFIKASI KONDISI RUMAH (Studi Kasus di Kabupaten Cilacap Tahun 2018)

2020 ◽  
Vol 9 (1) ◽  
pp. 41-49
Author(s):  
Johanes Roisa Prabowo ◽  
Rukun Santoso ◽  
Hasbi Yasin
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Conjugate Gradient ◽  
Training Data ◽  
Gradient Algorithm ◽  
Output Layer ◽  
Average Accuracy ◽  
Testing Data ◽  
Artificial Neural ◽  
Hidden Layer

House is one aspect of the welfare of society that must be met, because house is the main need for human life besides clothing and food. The condition of the house as a good shelter can be known from the structure and facilities of buildings. This research aims to analyze the classification of house conditions is livable or not livable. The method used is artificial neural networks (ANN). ANN is a system information processing that has characteristics similar to biological neural networks. In this research the optimization method used is the conjugate gradient algorithm. The data used are data of Survei Sosial Ekonomi Nasional (Susenas) March 2018 Kor Keterangan Perumahan for Cilacap Regency. The data is divided into training data and testing data with the proportion that gives the highest average accuracy is 90% for training data and 10% for testing data. The best architecture obtained a model consisting of 8 neurons in input layer, 10 neurons in hidden layer and 1 neuron in output layer. The activation function used are bipolar sigmoid in the hidden layer and binary sigmoid in the output layer. The results of the analysis showed that ANN works very well for classification on house conditions in Cilacap Regency with an average accuracy of 98.96% at the training stage and 97.58% at the testing stage.Keywords: House, Classification, Artificial Neural Networks, Conjugate Gradient

Sign in / Sign up

Export Citation Format

Share Document