Fast learning rates for regularized regression algorithms

2012 ◽  
Vol 42 (12) ◽  
pp. 1251-1262 ◽  
Author(s):  
HongZhi TONG ◽  
FengHong YANG ◽  
DiRong CHEN
Keyword(s):  
Regularized Regression ◽  
Fast Learning ◽  
Learning Rates ◽  
Regression Algorithms

LEARNING RATES OF REGULARIZED REGRESSION FOR FUNCTIONAL DATA

2009 ◽  
Vol 07 (06) ◽  
pp. 839-850 ◽  
Author(s):  
YONG-LI XU ◽  
DI-RONG CHEN
Keyword(s):  
Functional Data Analysis ◽  
Functional Data ◽  
Input Data ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Natural Extension ◽  
Least Square ◽  
Regularized Regression ◽  
Fast Learning ◽  
Learning Rates

The study of regularized learning algorithms is a very important issue and functional data analysis extends classical methods. We establish the learning rates of the least square regularized regression algorithm in reproducing kernel Hilbert space for functional data. With the iteration method, we obtain fast learning rate for functional data. Our result is a natural extension for least square regularized regression algorithm when the dimension of input data is finite.

Uncertainty Quantification and Optimization of Deep Learning for Fracture Recognition

2021 ◽  
Author(s):  
Ryan Santoso ◽  
Xupeng He ◽  
Marwa Alsinan ◽  
Hyung Kwak ◽  
Hussein Hoteit
Keyword(s):  
Deep Learning ◽  
Uncertainty Quantification ◽  
Learning Rate ◽  
Batch Size ◽  
Global Maximum ◽  
Fractured Reservoir ◽  
Training Set ◽  
Fast Learning ◽  
Learning Rates ◽  
Aleatoric Uncertainty

Abstract Automatic fracture recognition from borehole images or outcrops is applicable for the construction of fractured reservoir models. Deep learning for fracture recognition is subject to uncertainty due to sparse and imbalanced training set, and random initialization. We present a new workflow to optimize a deep learning model under uncertainty using U-Net. We consider both epistemic and aleatoric uncertainty of the model. We propose a U-Net architecture by inserting dropout layer after every "weighting" layer. We vary the dropout probability to investigate its impact on the uncertainty response. We build the training set and assign uniform distribution for each training parameter, such as the number of epochs, batch size, and learning rate. We then perform uncertainty quantification by running the model multiple times for each realization, where we capture the aleatoric response. In this approach, which is based on Monte Carlo Dropout, the variance map and F1-scores are utilized to evaluate the need to craft additional augmentations or stop the process. This work demonstrates the existence of uncertainty within the deep learning caused by sparse and imbalanced training sets. This issue leads to unstable predictions. The overall responses are accommodated in the form of aleatoric uncertainty. Our workflow utilizes the uncertainty response (variance map) as a measure to craft additional augmentations in the training set. High variance in certain features denotes the need to add new augmented images containing the features, either through affine transformation (rotation, translation, and scaling) or utilizing similar images. The augmentation improves the accuracy of the prediction, reduces the variance prediction, and stabilizes the output. Architecture, number of epochs, batch size, and learning rate are optimized under a fixed-uncertain training set. We perform the optimization by searching the global maximum of accuracy after running multiple realizations. Besides the quality of the training set, the learning rate is the heavy-hitter in the optimization process. The selected learning rate controls the diffusion of information in the model. Under the imbalanced condition, fast learning rates cause the model to miss the main features. The other challenge in fracture recognition on a real outcrop is to optimally pick the parental images to generate the initial training set. We suggest picking images from multiple sides of the outcrop, which shows significant variations of the features. This technique is needed to avoid long iteration within the workflow. We introduce a new approach to address the uncertainties associated with the training process and with the physical problem. The proposed approach is general in concept and can be applied to various deep-learning problems in geoscience.

Fast learning rates for sparse quantile regression problem

2013 ◽  
Vol 108 ◽  
pp. 13-22 ◽  
Author(s):  
Shao-Gao Lv ◽  
Tie-Feng Ma ◽  
Liu Liu ◽  
Yun-Long Feng
Keyword(s):  
Quantile Regression ◽  
Regression Problem ◽  
Fast Learning ◽  
Learning Rates

scholarly journals Fast learning rates for plug-in classifiers

2007 ◽  
Vol 35 (2) ◽  
pp. 608-633 ◽  
Author(s):  
Jean-Yves Audibert ◽  
Alexandre B. Tsybakov
Keyword(s):  
Fast Learning ◽  
Learning Rates

The Generalization Performance of Regularized Regression Algorithms Based on Markov Sampling

2014 ◽  
Vol 44 (9) ◽  
pp. 1497-1507 ◽  
Author(s):  
Bin Zou ◽  
Yuan Yan Tang ◽  
Zongben Xu ◽  
Luoqing Li ◽  
Jie Xu ◽  
...  
Keyword(s):  
Regularized Regression ◽  
Generalization Performance ◽  
Regression Algorithms
Sign in / Sign up

Export Citation Format

Share Document