Reliable training of convolutional neural networks for GPR-based buried threat detection using the Adam optimizer and batch normalization

2019 ◽  
Author(s):  
Steven Jacobson ◽  
Daniël Reichman ◽  
Joel Bjornstad ◽  
Leslie M. Collins ◽  
Jordan M. Malof
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Threat Detection ◽  
Batch Normalization

scholarly journals Interpolating Convolutional Neural Networks Using Batch Normalization

2018 ◽  
pp. 591-606
Author(s):  
Gratianus Wesley Putra Data ◽  
Kirjon Ngu ◽  
David William Murray ◽  
Victor Adrian Prisacariu
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Batch Normalization

scholarly journals An Analysis on Very Deep Convolutional Neural Networks: Problems and Solutions

2021 ◽  
Vol 66 (1) ◽  
pp. 5
Author(s):  
T.-V. Pricope
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Convolutional Neural Networks ◽  
Computation Time ◽  
Deep Convolutional Neural Networks ◽  
Batch Normalization ◽  
Image Dataset ◽  
Problems And Solutions ◽  
Hidden Patterns

Neural Networks have become a powerful tool in computer vision because of the recent breakthroughs in computation time and model architecture. Very deep models allow for better deciphering of the hidden patterns in the data; however, training them successfully is not a trivial problem, because of the notorious vanishing/exploding gradient problem. We illustrate this problem on VGG models, with 8 and 38 hidden layers, on the CIFAR100 image dataset, where we visualize how the gradients evolve during training. We explore known solutions to this problem like Batch Normalization (BatchNorm) or Residual Networks (ResNets), explaining the theory behind them. Our experiments show that the deeper model suffers from the vanishing gradient problem, but BatchNorm and ResNets do solve it. The employed solutions slighly improve the performance of shallower models as well, yet, the fixed deeper models outperform them.  

Sign in / Sign up

Export Citation Format

Share Document