Predicting Fine-Grained Air Quality Based on Deep Neural Networks

2020 ◽  
pp. 1-1
Author(s):  
Xiuwen Yi ◽  
Zhewen Duan ◽  
Ruiyuan Li ◽  
Junbo Zhang ◽  
Tianrui Li ◽  
...  
Keyword(s):  
Neural Networks ◽  
Air Quality ◽  
Deep Neural Networks ◽  
Fine Grained

Exploiting potential of deep neural networks by layer-wise fine-grained parallelism

2020 ◽  
Vol 102 ◽  
pp. 210-221 ◽  
Author(s):  
Wenbin Jiang ◽  
Yangsong Zhang ◽  
Pai Liu ◽  
Jing Peng ◽  
Laurence T. Yang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Fine Grained ◽  
Fine Grained Parallelism

scholarly journals RALR: Random Amplify Learning Rates for Training Neural Networks

2021 ◽  
Vol 12 (1) ◽  
pp. 268
Author(s):  
Jiali Deng ◽  
Haigang Gong ◽  
Minghui Liu ◽  
Tianshu Xie ◽  
Xuan Cheng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Data Augmentation ◽  
Saddle Points ◽  
Learning Rate ◽  
Global Learning ◽  
Fine Grained ◽  
Learning Rates ◽  
Overall Performance ◽  
Classification Tasks

It has been shown that the learning rate is one of the most critical hyper-parameters for the overall performance of deep neural networks. In this paper, we propose a new method for setting the global learning rate, named random amplify learning rates (RALR), to improve the performance of any optimizer in training deep neural networks. Instead of monotonically decreasing the learning rate, we expect to escape saddle points or local minima by amplifying the learning rate between reasonable boundary values based on a given probability. Training with RALR rather than conventionally decreasing the learning rate achieves further improvement on networks’ performance without extra consumption. Remarkably, the RALR is complementary with state-of-the-art data augmentation and regularization methods. Besides, we empirically study its performance on image classification tasks, fine-grained classification tasks, object detection tasks, and machine translation tasks. Experiments demonstrate that RALR can bring a notable improvement while preventing overfitting when training deep neural networks. For example, the classification accuracy of ResNet-110 trained on the CIFAR-100 dataset using RALR achieves a 1.34% gain compared with ResNet-110 trained traditionally.

Sign in / Sign up

Export Citation Format

Share Document