scholarly journals Formation of Inter-Frame Deformation Field of Images Using Reverse Stochastic Gradient Estimation

2019 ◽  
Author(s):  
Alexander Tashlinskii ◽  
Pavel Smirnov
Keyword(s):  
Stochastic Gradient ◽  
Deformation Field ◽  
Gradient Estimation ◽  
Inter Frame

scholarly journals Convergence characteristics at stochastic estimation of image inter-frame deformations

2019 ◽  
pp. 114-120
Author(s):  
A G Tashlinskii ◽  
A V Zhukova ◽  
D G Kraus
Keyword(s):  
Mathematical Modeling ◽  
Confidence Interval ◽  
Convergence Rate ◽  
Probability Density ◽  
Mathematical Expectation ◽  
Threshold Value ◽  
Stochastic Gradient ◽  
Stochastic Estimation ◽  
Gradient Estimation ◽  
Inter Frame

Several approaches to the numerical description of image inter-frame geometric deformations parameters estimates behavior at iterations of non-identification relay stochastic gradient estimation are considered. The probability density of the Euclidean mismatch distance of estimates vector is chosen as an argument of the characteristics forming the numerical values. It made it possible to ensure invariance of research to the set of parameters of the used inter-frame geometric deformations model. The mathematical expectation, the probability of exceeding a given threshold value of the convergence rate and the confidence interval of the Euclidean mismatch distance were investigated as characteristics. Probabilistic mathematical modeling is applied to calculate the probability density of the Euclidean mismatch distance.

scholarly journals PIXEL-BY-PIXEL ESTIMATION OF SCENE MOTION IN VIDEO

2017 ◽  
Vol XLII-2/W4 ◽  
pp. 61-65 ◽  
Author(s):  
A. G. Tashlinskii ◽  
P. V. Smirnov ◽  
M. G. Tsaryov
Keyword(s):  
Recursive Estimation ◽  
Stochastic Gradient ◽  
Moving Object ◽  
Stochastic Gradient Descent ◽  
Processing Efficiency ◽  
Estimated Parameters ◽  
Gradient Descent Algorithm ◽  
The Right ◽  
Moving Object Trajectory ◽  
Inter Frame

The paper considers the effectiveness of motion estimation in video using pixel-by-pixel recurrent algorithms. The algorithms use stochastic gradient decent to find inter-frame shifts of all pixels of a frame. These vectors form shift vectors’ field. As estimated parameters of the vectors the paper studies their projections and polar parameters. It considers two methods for estimating shift vectors’ field. The first method uses stochastic gradient descent algorithm to sequentially process all nodes of the image row-by-row. It processes each row bidirectionally i.e. from the left to the right and from the right to the left. Subsequent joint processing of the results allows compensating inertia of the recursive estimation. The second method uses correlation between rows to increase processing efficiency. It processes rows one after the other with the change in direction after each row and uses obtained values to form resulting estimate. The paper studies two criteria of its formation: gradient estimation minimum and correlation coefficient maximum. The paper gives examples of experimental results of pixel-by-pixel estimation for a video with a moving object and estimation of a moving object trajectory using shift vectors’ field.

Stochastic Gradient Estimation for Artificial Neural Networks

2019 ◽  
Author(s):  
Yijie Peng ◽  
Li Xiao ◽  
Bernd Heidergott ◽  
L. Jeff Hong ◽  
Henry Lam
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Stochastic Gradient ◽  
Gradient Estimation ◽  
Artificial Neural

A New Likelihood Ratio Method for Training Artificial Neural Networks

2021 ◽  
Author(s):  
Yijie Peng ◽  
Li Xiao ◽  
Bernd Heidergott ◽  
L. Jeff Hong ◽  
Henry Lam
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Likelihood Ratio ◽  
Stochastic Gradient ◽  
Ratio Method ◽  
Estimation Technique ◽  
Gradient Estimation ◽  
Backpropagation Method ◽  
Infinitesimal Perturbation ◽  
Artificial Neural

We investigate a new approach to compute the gradients of artificial neural networks (ANNs), based on the so-called push-out likelihood ratio method. Unlike the widely used backpropagation (BP) method that requires continuity of the loss function and the activation function, our approach bypasses this requirement by injecting artificial noises into the signals passed along the neurons. We show how this approach has a similar computational complexity as BP, and moreover is more advantageous in terms of removing the backward recursion and eliciting transparent formulas. We also formalize the connection between BP, a pivotal technique for training ANNs, and infinitesimal perturbation analysis, a classic path-wise derivative estimation approach, so that both our new proposed methods and BP can be better understood in the context of stochastic gradient estimation. Our approach allows efficient training for ANNs with more flexibility on the loss and activation functions, and shows empirical improvements on the robustness of ANNs under adversarial attacks and corruptions of natural noises. Summary of Contribution: Stochastic gradient estimation has been studied actively in simulation for decades and becomes more important in the era of machine learning and artificial intelligence. The stochastic gradient descent is a standard technique for training the artificial neural networks (ANNs), a pivotal problem in deep learning. The most popular stochastic gradient estimation technique is the backpropagation method. We find that the backpropagation method lies in the family of infinitesimal perturbation analysis, a path-wise gradient estimation technique in simulation. Moreover, we develop a new likelihood ratio-based method, another popular family of gradient estimation technique in simulation, for training more general ANNs, and demonstrate that the new training method can improve the robustness of the ANN.

Sign in / Sign up

Export Citation Format

Share Document