Adjoint- and Hybrid-Based Hessians for Optimization Problems in System Identification

2018 ◽  
Vol 140 (10) ◽  
Author(s):  
Souransu Nandi ◽  
Tarunraj Singh
Keyword(s):  
System Identification ◽  
Optimization Problems ◽  
Initial Conditions ◽  
Synthetic Data ◽  
Model Parameters ◽  
Computationally Efficient ◽  
Step Size ◽  
Adjoint Sensitivity ◽  
Gradient Based ◽  
Discrete Measurement

An adjoint sensitivity-based approach to determine the gradient and Hessian of cost functions for system identification of dynamical systems is presented. The motivation is the development of a computationally efficient approach relative to the direct differentiation (DD) technique and which overcomes the challenges of the step-size selection in finite difference (FD) approaches. An optimization framework is used to determine the parameters of a dynamical system which minimizes a summation of a scalar cost function evaluated at the discrete measurement instants. The discrete time measurements result in discontinuities in the Lagrange multipliers. Two approaches labeled as the Adjoint and the Hybrid are developed for the calculation of the gradient and Hessian for gradient-based optimization algorithms. The proposed approach is illustrated on the Lorenz 63 model where part of the initial conditions and model parameters are estimated using synthetic data. Examples of identifying model parameters of light curves of type 1a supernovae and a two-tank dynamic model using publicly available data are also included.

scholarly journals Sensitivity analysis and parameter estimation for the distributed modeling of infiltration excess overland flow

2007 ◽  
Vol 4 (1) ◽  
pp. 363-405 ◽  
Author(s):  
W. Castaings ◽  
D. Dartus ◽  
F.-X. Le Dimet ◽  
G.-M. Saulnier
Keyword(s):  
Sensitivity Analysis ◽  
Parameter Estimation ◽  
Overland Flow ◽  
Flash Flood ◽  
Parametric Uncertainty ◽  
Optimization Techniques ◽  
Model Parameters ◽  
Adjoint Sensitivity ◽  
Gradient Based ◽  
Infiltration Excess

Abstract. The variational methods widely used for other environmental systems are applied to a spatially distributed flash flood model coupling kinematic wave overland flow and Green Ampt infiltration. Using an idealized configuration where only parametric uncertainty is addressed, the potential of this approach is illustrated for sensitivity analysis and parameter estimation. Adjoint sensitivity analysis provides an extensive insight into the relation between model parameters and the hydrological response and enables the use of efficient gradient based optimization techniques.

Application of Differential Evolution in System Identification of Servo-Hydraulic System With a Flexible Load

2006 ◽  
Author(s):  
H. Yousefi ◽  
H. Handroos
Keyword(s):  
System Identification ◽  
Differential Evolution ◽  
Hydraulic System ◽  
Optimization Problems ◽  
Initial Conditions ◽  
Large Set ◽  
Unknown Parameters ◽  
Nonlinear Constraint ◽  
De Algorithm ◽  
Flexible Load

Electro Hydraulic Servo Systems (EHSS) with an asymmetrical cylinder are commonly used in industry. These kinds of systems are nonlinear in nature and their dynamic equations have several unknown parameters. System identification is a prerequisite to analysis of a dynamic system and design of an appropriate controller for improving its performance. In conventional identification methods, a model structure is selected and the parameters of that model are calculated by optimizing an objective function. This process usually requires a large set of input/output data from the system. In addition, the obtained parameters may be only locally optimal. One of the most promising novel evolutionary algorithms is the Differential Evolution (DE) algorithm for solving global optimization problems with continuous parameters. In this article, the DE algorithm is proposed for handling nonlinear constraint functions with boundary limits of variables to find the best parameters of a nonlinear servo-hydraulic system with flexible load. The DE guarantees Fast speed convergence and accurate solutions regardless the initial conditions of parameters. The results suggest that, DE is useful, reliable and easy to use tools in many aspects of control engineering and especially in system identification.

scholarly journals Information-Maximization Clustering Based on Squared-Loss Mutual Information

2014 ◽  
Vol 26 (1) ◽  
pp. 84-131 ◽  
Author(s):  
Masashi Sugiyama ◽  
Gang Niu ◽  
Makoto Yamada ◽  
Manabu Kimura ◽  
Hirotaka Hachiya
Keyword(s):  
Mutual Information ◽  
Optimization Problems ◽  
Continuous Optimization ◽  
Optimal Solution ◽  
Selection Procedure ◽  
Eigenvalue Decomposition ◽  
Model Parameters ◽  
Computationally Efficient ◽  
Novel Approach ◽  
Information Maximization

Information-maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it involves only continuous optimization of model parameters, which is substantially simpler than discrete optimization of cluster assignments. However, existing methods still involve nonconvex optimization problems, and therefore finding a good local optimal solution is not straightforward in practice. In this letter, we propose an alternative information-maximization clustering method based on a squared-loss variant of mutual information. This novel approach gives a clustering solution analytically in a computationally efficient way via kernel eigenvalue decomposition. Furthermore, we provide a practical model selection procedure that allows us to objectively optimize tuning parameters included in the kernel function. Through experiments, we demonstrate the usefulness of the proposed approach.

Computational Improvements to Estimating Kriging Metamodel Parameters

2009 ◽  
Vol 131 (8) ◽  
Author(s):  
Jay D. Martin
Keyword(s):  
Response Surface ◽  
Spatial Correlation ◽  
Likelihood Estimation ◽  
Kriging Model ◽  
Model Parameters ◽  
Computationally Efficient ◽  
Spatial Correlation Function ◽  
Computational Burden ◽  
Kriging Metamodel ◽  
Gradient Based

The details of a method to reduce the computational burden experienced while estimating the optimal model parameters for a Kriging model are presented. A Kriging model is a type of surrogate model that can be used to create a response surface based a set of observations of a computationally expensive system design analysis. This Kriging model can then be used as a computationally efficient surrogate to the original model, providing the opportunity for the rapid exploration of the resulting tradespace. The Kriging model can provide a more complex response surface than the more traditional linear regression response surface through the introduction of a few terms to quantify the spatial correlation of the observations. Implementation details and enhancements to gradient-based methods to estimate the model parameters are presented. It concludes with a comparison of these enhancements to using maximum likelihood estimation to estimate Kriging model parameters and their potential reduction in computational burden. These enhancements include the development of the analytic gradient and Hessian for the log-likelihood equation of a Kriging model that uses a Gaussian spatial correlation function. The suggested algorithm is similar to the SCORING algorithm traditionally used in statistics.

Accounting for Parametric Model Uncertainty in Collision Avoidance for Unmanned Vehicles Using Sparse Grid Interpolation

2013 ◽  
Author(s):  
Sarah L. Noble ◽  
Joel M. Esposito ◽  
Jason Case
Keyword(s):  
Collision Avoidance ◽  
Ad Hoc ◽  
Initial Conditions ◽  
Interpolation Method ◽  
Unmanned Vehicles ◽  
Sparse Grid ◽  
Model Parameters ◽  
Ground Vehicles ◽  
Computationally Efficient ◽  
Grid Interpolation

In this paper we present an enhancement of model-based trajectory selection algorithms — a popular class of collision avoidance techniques for autonomous ground vehicles. Rather than dilate a set of individual candidate trajectories in an ad hoc way to account for uncertainty, we generate a set of trajectory clouds — sets of states that represent possible future poses over a product of intervals representing uncertainty in the model parameters, initial conditions and actuator commands. The clouds are generated using the sparse-grid interpolation method which is both error-controlled and computationally efficient. The approach is implemented on a differential drive vehicle.

Neural Dynamics and Newton–Raphson Iteration for Nonlinear Optimization

2014 ◽  
Vol 9 (2) ◽  
Author(s):  
Dongsheng Guo ◽  
Yunong Zhang
Keyword(s):  
Nonlinear Optimization ◽  
Optimization Problems ◽  
Activation Function ◽  
Neural Dynamics ◽  
Time Varying ◽  
Step Size ◽  
Gradient Based ◽  
Digital Hardware ◽  
Newton Raphson ◽  
Raphson Iteration

In this paper, a special type of neural dynamics (ND) is generalized and investigated for time-varying and static scalar-valued nonlinear optimization. In addition, for comparative purpose, the gradient-based neural dynamics (or termed gradient dynamics (GD)) is studied for nonlinear optimization. Moreover, for possible digital hardware realization, discrete-time ND (DTND) models are developed. With the linear activation function used and with the step size being 1, the DTND model reduces to Newton–Raphson iteration (NRI) for solving the static nonlinear optimization problems. That is, the well-known NRI method can be viewed as a special case of the DTND model. Besides, the geometric representation of the ND models is given for time-varying nonlinear optimization. Numerical results demonstrate the efficacy and advantages of the proposed ND models for time-varying and static nonlinear optimization.

scholarly journals A Novel Solution Methodology Based on a Modified Gradient-Based Optimizer for Parameter Estimation of Photovoltaic Models

Electronics ◽  
2021 ◽  
Vol 10 (4) ◽  
pp. 472
Author(s):  
Mohamed H. Hassan ◽  
Salah Kamel ◽  
M. A. El-Dabah ◽  
Hegazy Rezk
Keyword(s):  
Optimization Problems ◽  
Parameters Estimation ◽  
Model Parameters ◽  
Decision Variable ◽  
Unknown Parameters ◽  
Local Optima ◽  
Pv Module ◽  
Squared Error ◽  
Gradient Based ◽  
Solution Methodology

In this paper, a modified version of a recent optimization algorithm called gradient-based optimizer (GBO) is proposed with the aim of improving its performance. Both the original gradient-based optimizer and the modified version, MGBO, are utilized for estimating the parameters of Photovoltaic models. The MGBO has the advantages of accelerated convergence rate as well as avoiding the local optima. These features make it compatible for investigating its performance in one of the nonlinear optimization problems like Photovoltaic model parameters estimation. The MGBO is used for the identification of parameters of different Photovoltaic models; single-diode, double-diode, and PV module. To obtain a generic Photovoltaic model, it is required to fit the experimentally obtained data. During the optimization process, the unknown parameters of the PV model are used as a decision variable whereas the root means squared error between the measured and estimated data is used as a cost function. The results verified the fast conversion rate and precision of the MGBO over other recently reported algorithms in solving the studied optimization problem.

DynManto: A Matlab Toolbox for the Simulation and Analysis of Multibody Systems

2020 ◽  
Author(s):  
Alexander Held ◽  
Ali Moghadasi ◽  
Robert Seifried
Keyword(s):  
Sensitivity Analysis ◽  
Multibody Systems ◽  
Equations Of Motion ◽  
The Body ◽  
Computationally Efficient ◽  
Adjoint Sensitivity ◽  
Modeling And Analysis ◽  
Gradient Based ◽  
Data Files ◽  
Parameter Identifications

Abstract The Dynamic Modeling and Analysis Toolbox DynManto is an acedemic Matlab code which allows the modeling, simulation and sensitivity analysis of spatial multibody systems. The kinematics of rigid and flexible bodies is described by the floating frame of reference formulation and the body properties are provided by standard input data files. In this way the evaluation of the equations of motion is computationally efficient and an arbitrary parameterization of the system can be achieved. The latter is important in the automated adjoint sensitivity analysis of multibody systems, which yields gradient information for system analyses, parameter identifications or gradient-based optimizations. The capabilities of DynManto are demonstrated by the application examples of a flexible two-arm manipulator and Chebyshev’s Lambda Mechanism.

A Comprehensive Investigation of Ensembles of Gaussian-Based and Gradient-Based Transformed Reliability Analyses: When and How to Use Them

2016 ◽  
Author(s):  
Po Ting Lin ◽  
Wei-Hao Lu ◽  
Shu-Ping Lin
Keyword(s):  
Design Optimization ◽  
Design Space ◽  
Optimization Problems ◽  
Nonlinear Problems ◽  
Kernel Functions ◽  
Sensitivity Analyses ◽  
Normal Space ◽  
Highly Nonlinear ◽  
Standard Normal ◽  
Gradient Based

In the past few years, researchers have begun to investigate the existence of arbitrary uncertainties in the design optimization problems. Most traditional reliability-based design optimization (RBDO) methods transform the design space to the standard normal space for reliability analysis but may not work well when the random variables are arbitrarily distributed. It is because that the transformation to the standard normal space cannot be determined or the distribution type is unknown. The methods of Ensemble of Gaussian-based Reliability Analyses (EoGRA) and Ensemble of Gradient-based Transformed Reliability Analyses (EGTRA) have been developed to estimate the joint probability density function using the ensemble of kernel functions. EoGRA performs a series of Gaussian-based kernel reliability analyses and merged them together to compute the reliability of the design point. EGTRA transforms the design space to the single-variate design space toward the constraint gradient, where the kernel reliability analyses become much less costly. In this paper, a series of comprehensive investigations were performed to study the similarities and differences between EoGRA and EGTRA. The results showed that EGTRA performs accurate and effective reliability analyses for both linear and nonlinear problems. When the constraints are highly nonlinear, EGTRA may have little problem but still can be effective in terms of starting from deterministic optimal points. On the other hands, the sensitivity analyses of EoGRA may be ineffective when the random distribution is completely inside the feasible space or infeasible space. However, EoGRA can find acceptable design points when starting from deterministic optimal points. Moreover, EoGRA is capable of delivering estimated failure probability of each constraint during the optimization processes, which may be convenient for some applications.

Sign in / Sign up

Export Citation Format

Share Document