scholarly journals Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization

2020 ◽  
Vol 10 (8) ◽  
pp. 2950
Author(s):  
Qiuyu Zhu ◽  
Zikuang He ◽  
Tao Zhang ◽  
Wennan Cui
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Back Propagation ◽  
Ground Truth ◽  
Classification Performance ◽  
Batch Normalization ◽  
Network Output ◽  
Gradient Calculation ◽  
Output Probability ◽  
Better Than

Convolutional neural networks (CNNs) have made great achievements on computer vision tasks, especially the image classification. With the improvement of network structure and loss functions, the performance of image classification is getting higher and higher. The classic Softmax + cross-entropy loss has been the norm for training neural networks for years, which is calculated from the output probability of the ground-truth class. Then the network’s weight is updated by gradient calculation of the loss. However, after several epochs of training, the back-propagation errors usually become almost negligible. For the above considerations, we proposed that batch normalization with adjustable scale could be added after network output to alleviate the problem of vanishing gradient problem in deep learning. The experimental results show that our method can significantly improve the final classification accuracy on different network structures, and is also better than many other improved classification Loss.

DCNN-based Ship Classification using Enhanced Edge Information and Inception Module

2021 ◽  
Author(s):  
Bo Wang ◽  
Xiaoting Yu ◽  
Chengeng Huang ◽  
Qinghong Sheng ◽  
Yuanyuan Wang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Classification Performance ◽  
Image Features ◽  
Deep Convolutional Neural Networks ◽  
Edge Information ◽  
Average Accuracy ◽  
Ship Classification ◽  
Edge Features ◽  
High Level ◽  
Better Than

The excellent feature extraction ability of deep convolutional neural networks (DCNNs) has been demonstrated in many image processing tasks, by which image classification can achieve high accuracy with only raw input images. However, the specific image features that influence the classification results are not readily determinable and what lies behind the predictions is unclear. This study proposes a method combining the Sobel and Canny operators and an Inception module for ship classification. The Sobel and Canny operators obtain enhanced edge features from the input images. A convolutional layer is replaced with the Inception module, which can automatically select the proper convolution kernel for ship objects in different image regions. The principle is that the high-level features abstracted by the DCNN, and the features obtained by multi-convolution concatenation of the Inception module must ultimately derive from the edge information of the preprocessing input images. This indicates that the classification results are based on the input edge features, which indirectly interpret the classification results to some extent. Experimental results show that the combination of the edge features and the Inception module improves DCNN ship classification performance. The original model with the raw dataset has an average accuracy of 88.72%, while when using enhanced edge features as input, it achieves the best performance of 90.54% among all models. The model that replaces the fifth convolutional layer with the Inception module has the best performance of 89.50%. It performs close to VGG-16 on the raw dataset and is significantly better than other deep neural networks. The results validate the functionality and feasibility of the idea posited.

scholarly journals Unsupervised Learning Predicts Human Perception and Misperception of Gloss

2020 ◽  
Author(s):  
Katherine R. Storrs ◽  
Roland W. Fleming
Keyword(s):  
Neural Networks ◽  
Unsupervised Learning ◽  
Human Perception ◽  
Ground Truth ◽  
Statistical Structure ◽  
Specular Reflectance ◽  
Explicit Information ◽  
Control Models ◽  
Internal Code ◽  
Better Than

AbstractReflectance, lighting, and geometry combine in complex ways to create images. How do we disentangle these to perceive individual properties, like the glossiness of a surface? We suggest that brains disentangle properties by learning to model statistical structure in proximal images. To test this, we trained unsupervised generative neural networks on renderings of glossy surfaces and compared their representations with human gloss judgments. The networks spontaneously cluster images according to distal properties such as reflectance and illumination, despite receiving no explicit information about them. Intriguingly, the resulting representations predict the specific patterns of ‘successes’ and ‘errors’ in human perception. Linearly decoding specular reflectance from the model’s internal code predicts human gloss perception better than ground truth, supervised networks, or control models, and predicts, on an image-by-image basis, illusions of gloss perception caused by interactions between material, shape, and lighting. Unsupervised learning may underlie many perceptual dimensions in vision, and beyond.

scholarly journals Comparison of Classical Computer Vision vs. Convolutional Neural Networks for Weed Mapping in Aerial Images

2020 ◽  
Vol 27 (4) ◽  
pp. 20-33
Author(s):  
Paulo César Pereira Júnior ◽  
Alexandre Monteiro ◽  
Rafael Da Luz Ribeiro ◽  
Antonio Carlos Sobieranski ◽  
Aldo Von Wangenheim
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Convolutional Neural Networks ◽  
Precision Agriculture ◽  
Ground Truth ◽  
Aerial Images ◽  
Weed Mapping ◽  
Classical Models ◽  
Classical Computer ◽  
Better Than

In this paper, we present a comparison between convolutional neural networks and classicalcomputer vision approaches, for the specific precision agriculture problem of weed mapping on sugarcane fields aerial images. A systematic literature review was conducted to find which computer vision methods are being used on this specific problem. The most cited methods were implemented, as well as four models of convolutional neural networks. All implemented approaches were tested using the same dataset, and their results were quantitatively and qualitatively analyzed. The obtained results were compared to a human expert made ground truth, for validation. The results indicate that the convolutional neural networks present better precision and generalize better than the classical models

scholarly journals A Machine Learning based Approach for Segmenting Retinal Nerve Images using Artificial Neural Networks

2020 ◽  
Vol 10 (4) ◽  
pp. 5986-5991
Author(s):  
A. N. Saeed
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Diabetic Retinopathy ◽  
Blood Vessels ◽  
Back Propagation ◽  
Texture Feature ◽  
Texture Features ◽  
Classification Performance ◽  
Fundus Images ◽  
Retinal Blood Vessels

Artificial Intelligence (AI) based Machine Learning (ML) is gaining more attention from researchers. In ophthalmology, ML has been applied to fundus photographs, achieving robust classification performance in the detection of diseases such as diabetic retinopathy, retinopathy of prematurity, etc. The detection and extraction of blood vessels in the retina is an essential part of various diagnosing problems associated with eyes, such as diabetic retinopathy. This paper proposes a novel machine learning approach to segment the retinal blood vessels from eye fundus images using a combination of color features, texture features, and Back Propagation Neural Networks (BPNN). The proposed method comprises of two steps, namely the color texture feature extraction and training the BPNN to get the segmented retinal nerves. Magenta color and correlation-texture features are given as input to the BPNN. The system was trained and tested in retinal fundus images taken from two distinct databases. The average sensitivity, specificity, and accuracy obtained for the segmentation of retinal blood vessels were 0.470%, 0.914%, and 0.903% respectively. Results obtained reveal that the proposed methodology is excellent in automated segmentation retinal nerves. The proposed segmentation methodology was able to obtain comparable accuracy with other methods.

scholarly journals Back-propagation learning in deep Spike-By-Spike networks

2019 ◽  
Author(s):  
David Rotermund ◽  
Klaus R. Pawelzik
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Action Potentials ◽  
Back Propagation ◽  
Learning Rule ◽  
Building Blocks ◽  
Classification Performance ◽  
Generative Models ◽  
Learning Method ◽  
Discrete Action

ABSTRACTNeural networks are important building blocks in technical applications. These artificial neural networks (ANNs) rely on noiseless continuous signals in stark contrast to the discrete action potentials stochastically exchanged among the neurons in real brains. A promising approach towards bridging this gap are the Spike-by-Spike (SbS) networks which represent a compromise between non-spiking and spiking versions of generative models that perform inference on their inputs. What is still missing are algorithms for finding weight sets that would optimize the output performances of deep SbS networks with many layers.Here, a learning rule for hierarchically organized SbS networks is derived. The properties of this approach are investigated and its functionality demonstrated by simulations. In particular, a Deep Convolutional SbS network for classifying handwritten digits (MNIST) is presented. When applied together with an optimizer this learning method achieves a classification performance of roughly 99.3% on the MNIST test data. Thereby it approaches the benchmark results of ANNs without extensive parameter optimization. We envision that with this learning rule SBS networks will provide a new basis for research in neuroscience and for technical applications, especially when they become implemented on specialized computational hardware.

scholarly journals Tessarine and Quaternion-Valued Deep Neural Networks for Image Classification

2021 ◽  
Author(s):  
Fernando Ribeiro de Senna ◽  
Marcos Eduardo Valle
Keyword(s):  
Neural Networks ◽  
Image Processing ◽  
Image Classification ◽  
Deep Neural Networks ◽  
Real Numbers ◽  
Image Processing And Analysis ◽  
Batch Normalization

Many image processing and analysis tasks are performed with deep neural networks. Although the vast majority of advances have been made with real numbers, recent works have shown that complex and hypercomplex-valued networks may achieve better results. In this paper, we address quaternion-valued and introduce tessarine-valued deep neural networks, including tessarine-valued 2D convolutions. We also address initialization schemes and hypercomplex batch normalization. Finally, a tessarine-valued ResNet model with hypercomplex batch normalization outperformed the corresponding real and quaternion-valued networks on the CIFAR dataset.

Endek Classification Based On GLCM Using Artificial Neural Networks with Adam Optimization

2020 ◽  
Vol 9 (2) ◽  
pp. 285
Author(s):  
Putu Wahyu Tirta Guna ◽  
Luh Arida Ayu Ayu Rahning Putri
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Back Propagation ◽  
Classification Problem ◽  
Model Accuracy ◽  
Computing Power ◽  
The Neural Network ◽  
Extraction Algorithm ◽  
Better Than

Not many people know that endek cloth itself has 4 known variances. .Nowadays. Computing and classification algorithm can be implemented to solve classification problem with respect to the features data as input. We can use this computing power to digitalize these endek pattern. The features extraction algorithm used in this research is GLCM. Where these data will act as input for the neural network model later. There is a lot of optimizer algorithm to use in back propagation phase. In this research we  prefer to use adam which is one of the newest and most popular optimizer algorithm. To compare its performace we also use SGD which is older and popular optimizer algorithm. Later we find that adam algorithm generate 33% accuracy which is better than what SGD algorithm give, it is 23% accuracy. Longer epoch also give affect for overall model accuracy.

A new neural network approach to density calculation on ceramic materials

2021 ◽  
Author(s):  
Vojislav V. Mitic ◽  
Srdjan Ribar ◽  
Branislav M. Randjelovic ◽  
Dejan Aleksic ◽  
Hans Fecht ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Error Control ◽  
Back Propagation ◽  
Ceramic Materials ◽  
Training Procedure ◽  
Neural Network Approach ◽  
Network Output ◽  
To Come ◽  
Error Generation

The materials’ consolidation, especially ceramics, is very important in advanced research development and industrial technologies. Science of sintering with all incoming novelties is the base of all these processes. A very important question in all of this is how to get the more precise structure parameters within the morphology of different ceramic materials. In that sense, the advanced procedure in collecting precise data in submicro-processes is also in direction of advanced miniaturization. Our research, based on different electrophysical parameters, like relative capacitance, breakdown voltage, and [Formula: see text], has been used in neural networks and graph theory successful applications. We extended furthermore our neural network back propagation (BP) on sintering parameters’ data. Prognosed mapping we can succeed if we use the coefficients, implemented by the training procedure. In this paper, we continue to apply the novelty from the previous research, where the error is calculated as a difference between the designed and actual network output. So, the weight coefficients contribute in error generation. We used the experimental data of sintered materials’ density, measured and calculated in the bulk, and developed possibility to calculate the materials’ density inside of consolidated structures. The BP procedure here is like a tool to come down between the layers, with much more precise materials’ density, in the points on morphology, which are interesting for different microstructure developments and applications. We practically replaced the errors’ network by density values, from ceramic consolidation. Our neural networks’ application novelty is successfully applied within the experimental ceramic material density [Formula: see text] [kg/m3], confirming the direction way to implement this procedure in other density cases. There are many different mathematical tools or tools from the field of artificial intelligence that can be used in such or similar applications. We choose to use artificial neural networks because of their simplicity and their self-improvement process, through BP error control. All of this contributes to the great improvement in the whole research and science of sintering technology, which is important for collecting more efficient and faster results.

Neural Network-Based Geometry Classification for Navigation Satellite Selection

2003 ◽  
Vol 56 (2) ◽  
pp. 291-304 ◽  
Author(s):  
Dah-Jing Jwo ◽  
Chien-Cheng Lai
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Probabilistic Neural Network ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Classification Performance ◽  
Navigation Satellite ◽  
Training Time ◽  
Fast Learning ◽  
Satellite Geometry

The neural networks (NN)-based geometry classification for good or acceptable navigation satellite subset selection is presented. The approach is based on classifying the values of satellite Geometry Dilution of Precision (GDOP) utilizing the classification-type NNs. Unlike some of the NNs that approximate the function, such as the back-propagation neural network (BPNN), the NNs here are employed as classifiers. Although BPNN can also be employed as a classifier, it takes a long training time. Two other methods that feature a fast learning speed will be implemented, including Optimal Interpolative (OI) Net and Probabilistic Neural Network (PNN). Simulation results from these three neural networks are presented. The classification performance and computational expense of neural network-based GDOP classification are explored.

Sign in / Sign up

Export Citation Format

Share Document