Remark on Algorithm 702—the updated truncated Newton minimization package

1999 ◽  
Vol 25 (1) ◽  
pp. 108-122 ◽  
Author(s):  
Dexuan Xie ◽  
Tamar Schlick
Keyword(s):  
Truncated Newton ◽  
Newton Minimization

scholarly journals A New Hyper-Parameter Optimization Method for Power Load Forecast Based on Recurrent Neural Networks

Algorithms ◽  
2021 ◽  
Vol 14 (6) ◽  
pp. 163
Author(s):  
Yaru Li ◽  
Yulai Zhang ◽  
Yongping Cai
Keyword(s):  
Neural Networks ◽  
Parameter Optimization ◽  
Recurrent Neural Networks ◽  
Critical Role ◽  
Optimization Method ◽  
Bayesian Optimization ◽  
Power Load ◽  
Gradient Calculation ◽  
Truncated Newton ◽  
Selection Of

The selection of the hyper-parameters plays a critical role in the task of prediction based on the recurrent neural networks (RNN). Traditionally, the hyper-parameters of the machine learning models are selected by simulations as well as human experiences. In recent years, multiple algorithms based on Bayesian optimization (BO) are developed to determine the optimal values of the hyper-parameters. In most of these methods, gradients are required to be calculated. In this work, the particle swarm optimization (PSO) is used under the BO framework to develop a new method for hyper-parameter optimization. The proposed algorithm (BO-PSO) is free of gradient calculation and the particles can be optimized in parallel naturally. So the computational complexity can be effectively reduced which means better hyper-parameters can be obtained under the same amount of calculation. Experiments are done on real world power load data,where the proposed method outperforms the existing state-of-the-art algorithms,BO with limit-BFGS-bound (BO-L-BFGS-B) and BO with truncated-newton (BO-TNC),in terms of the prediction accuracy. The errors of the prediction result in different models show that BO-PSO is an effective hyper-parameter optimization method.

scholarly journals Parsimonious truncated Newton method for time-domain full waveform inversion based on Fourier-domain full-scattered-field approximation

Geophysics ◽  
2021 ◽  
pp. 1-147
Author(s):  
Peng Yong ◽  
Romain Brossier ◽  
Ludovic Métivier
Keyword(s):  
Frequency Domain ◽  
Time Domain ◽  
Cross Correlation ◽  
Waveform Inversion ◽  
Field Approximation ◽  
Second Order ◽  
Vector Product ◽  
First Order ◽  
Full Waveform ◽  
Truncated Newton

In order to exploit Hessian information in Full Waveform Inversion (FWI), the matrix-free truncated Newton method can be used. In such a method, Hessian-vector product computation is one of the major concerns due to the huge memory requirements and demanding computational cost. Using the adjoint-state method, the Hessian-vector product can be estimated by zero-lag cross-correlation of the first-order/second-order incident wavefields and the second-order/first-order adjoint wavefields. Different from the implementation in frequency-domain FWI, Hessian-vector product construction in the time domain becomes much more challenging as it is not affordable to store the entire time-dependent wavefields. The widely used wavefield recomputation strategy leads to computationally intensive tasks. We present an efficient alternative approach to computing the Hessian-vector product for time-domain FWI. In our method, discrete Fourier transform is applied to extract frequency-domain components of involved wavefields, which are used to compute wavefield cross-correlation in the frequency domain. This makes it possible to avoid reconstructing the first-order and second-order incident wavefields. In addition, a full-scattered-field approximation is proposed to efficiently simplify the second-order incident and adjoint wavefields computation, which enables us to refrain from repeatedly solving the first-order incident and adjoint equations for the second-order incident and adjoint wavefields (re)computation. With the proposed method, the computational time can be reduced by 70% and 80% in viscous media for Gauss-Newton and full-Newton Hessian-vector product construction, respectively. The effectiveness of our method is also verified in the frame of a 2D multi-parameter inversion, in which the proposed method almost reaches the same iterative convergence of the conventional time-domain implementation.

Sign in / Sign up

Export Citation Format

Share Document