scholarly journals The Convex Information Bottleneck Lagrangian

Entropy ◽  
2020 ◽  
Vol 22 (1) ◽  
pp. 98 ◽  
Author(s):  
Borja Rodríguez Gálvez ◽  
Ragnar Thobaben ◽  
Mikael Skoglund
Keyword(s):  
Lagrange Multiplier ◽  
Optimization Problem ◽  
Random Variable ◽  
Practical Reasons ◽  
Compression Rate ◽  
Constrained Optimization Problem ◽  
Information Bottleneck ◽  
Constrained Problem ◽  
Deterministic Function ◽  
Compression Level

The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations T of some random variable X for the task of predicting Y. It is defined as a constrained optimization problem that maximizes the information the representation has about the task, I ( T ; Y ) , while ensuring that a certain level of compression r is achieved (i.e., I ( X ; T ) ≤ r ). For practical reasons, the problem is usually solved by maximizing the IB Lagrangian (i.e., L IB ( T ; β ) = I ( T ; Y ) − β I ( X ; T ) ) for many values of β ∈ [ 0 , 1 ] . Then, the curve of maximal I ( T ; Y ) for a given I ( X ; T ) is drawn and a representation with the desired predictability and compression is selected. It is known when Y is a deterministic function of X, the IB curve cannot be explored and another Lagrangian has been proposed to tackle this problem: the squared IB Lagrangian: L sq − IB ( T ; β sq ) = I ( T ; Y ) − β sq I ( X ; T ) 2 . In this paper, we (i) present a general family of Lagrangians which allow for the exploration of the IB curve in all scenarios; (ii) provide the exact one-to-one mapping between the Lagrange multiplier and the desired compression rate r for known IB curve shapes; and (iii) show we can approximately obtain a specific compression level with the convex IB Lagrangian for both known and unknown IB curve shapes. This eliminates the burden of solving the optimization problem for many values of the Lagrange multiplier. That is, we prove that we can solve the original constrained problem with a single optimization.

Error Compensation Based on Lagrange Multiplier in Heterodyne Laser Interferometer

2014 ◽  
Vol 681 ◽  
pp. 43-46
Author(s):  
Eun Hwan Oh ◽  
Woo Ram Lee ◽  
Kyung Hyun Lee ◽  
Kwan Ho You
Keyword(s):  
Constrained Optimization ◽  
Objective Function ◽  
Lagrange Multiplier ◽  
Error Compensation ◽  
Optimization Problem ◽  
Laser Interferometer ◽  
Length Measurement ◽  
Constrained Optimization Problem ◽  
Compensation Algorithm ◽  
Heterodyne Laser Interferometer

In this paper, we propose a signal compensation algorithm. In heterodyne laser interferometer, the unexpected error restricts the precision such as nonlinearity and environmental error. To improve the accuracy in length measurement, we use the method of Lagrange multiplier which solves the constrained optimization problem and allows to minimize an objective function. With the method of Lagrange, we apply it to a length measurement and show the result of simulation.

scholarly journals Application of Lagrange Multiplier Method for Computing Fold Bifurcation Point in A Two-Prey One Predator Dynamical System

2018 ◽  
Vol 24 (2) ◽  
pp. 7-19
Author(s):  
Marwan Marwan ◽  
Johan Matheus Tuwankotta ◽  
Eric Harjanto
Keyword(s):  
Dynamical System ◽  
Lagrange Multiplier ◽  
Bifurcation Point ◽  
Optimization Problem ◽  
The Other ◽  
Lagrange Multiplier Method ◽  
Multiplier Method ◽  
Constrained Optimization Problem ◽  
Fold Bifurcation ◽  
The Cost

We propose by means of an example of applications of the classical Lagrange Multiplier Method for computing fold bifurcation point of an equilibrium ina one-parameter family of dynamical systems. We have used the fact that an equilibrium of a system, geometrically can be seen as an intersection between nullcline manifolds of the system. Thus, we can view the problem of two collapsing equilibria as a constrained optimization problem, where one of the nullclines acts as the cost function while the other nullclines act as the constraints.

scholarly journals Nonlinear Information Bottleneck

Entropy ◽  
2019 ◽  
Vol 21 (12) ◽  
pp. 1181 ◽  
Author(s):  
Artemy Kolchinsky ◽  
Brendan D. Tracey ◽  
David H. Wolpert
Keyword(s):  
Neural Networks ◽  
Joint Distribution ◽  
Optimization Problem ◽  
Random Variable ◽  
Small State ◽  
State Spaces ◽  
Information Bottleneck ◽  
Optimal Encoding ◽  
Real World Datasets ◽  
Ib Method

Information bottleneck (IB) is a technique for extracting information in one random variable X that is relevant for predicting another random variable Y. IB works by encoding X in a compressed “bottleneck” random variable M from which Y can be accurately decoded. However, finding the optimal bottleneck variable involves a difficult optimization problem, which until recently has been considered for only two limited cases: discrete X and Y with small state spaces, and continuous X and Y with a Gaussian joint distribution (in which case optimal encoding and decoding maps are linear). We propose a method for performing IB on arbitrarily-distributed discrete and/or continuous X and Y, while allowing for nonlinear encoding and decoding maps. Our approach relies on a novel non-parametric upper bound for mutual information. We describe how to implement our method using neural networks. We then show that it achieves better performance than the recently-proposed “variational IB” method on several real-world datasets.

scholarly journals 2-factor Newton method for solving the constrained optimization problem with the singular Kuhn—Tucker system

2019 ◽  
Vol 485 (1) ◽  
pp. 19-21
Author(s):  
Yu. G. Evtushenko ◽  
A. A. Tret’yakov
Keyword(s):  
Constrained Optimization ◽  
Optimality Conditions ◽  
Newton Method ◽  
Optimization Problem ◽  
Necessary Optimality Conditions ◽  
Degenerate System ◽  
Constrained Optimization Problem ◽  
Complementarity Conditions ◽  
Constrained Problem ◽  
Solution Point

A new method for solving the inequality constrained optimization problem is proposed for the case when the system of necessary optimality conditions of Kuhn—Tucker is degenerate. This situation occurs for example in the case when strict complementarity conditions fails in solution point. The reduction of the inequalities con- strained optimization problem to the equalities constrained problem is substantiated and the use of a new 2-fac- tor Newton method for the effective solution of the obtained degenerate system of optimality conditions is shown.

scholarly journals Nonconvex constrained optimization by a filtering branch and bound

2020 ◽  
Author(s):  
Gabriele Eichfelder ◽  
Kathrin Klamroth ◽  
Julia Niebling
Keyword(s):  
Constrained Optimization ◽  
Optimization Problem ◽  
Feasible Solution ◽  
Constrained Optimization Problem ◽  
Numerical Tests ◽  
Nonconvex Constrained Optimization ◽  
Objective Value ◽  
Feasible Solutions ◽  
Quality Guarantee ◽  
Single Objective

AbstractA major difficulty in optimization with nonconvex constraints is to find feasible solutions. As simple examples show, the $$\alpha $$ α BB-algorithm for single-objective optimization may fail to compute feasible solutions even though this algorithm is a popular method in global optimization. In this work, we introduce a filtering approach motivated by a multiobjective reformulation of the constrained optimization problem. Moreover, the multiobjective reformulation enables to identify the trade-off between constraint satisfaction and objective value which is also reflected in the quality guarantee. Numerical tests validate that we indeed can find feasible and often optimal solutions where the classical single-objective $$\alpha $$ α BB method fails, i.e., it terminates without ever finding a feasible solution.

scholarly journals Optimal P-Q Control of Grid-Connected Inverters in a Microgrid Based on Adaptive Population Extremal Optimization

Energies ◽  
2018 ◽  
Vol 11 (8) ◽  
pp. 2107 ◽  
Author(s):  
Min-Rong Chen ◽  
Huan Wang ◽  
Guo-Qiang Zeng ◽  
Yu-Xing Dai ◽  
Da-Qiang Bi
Keyword(s):  
Constrained Optimization ◽  
Optimization Problem ◽  
Reactive Power ◽  
Active Power ◽  
Adaptive Genetic Algorithm ◽  
Constrained Optimization Problem ◽  
Extremal Optimization ◽  
Active And Reactive Power ◽  
Three Phase ◽  
Grid Connected Inverter

The optimal P-Q control issue of the active and reactive power for a microgrid in the grid-connected mode has attracted increasing interests recently. In this paper, an optimal active and reactive power control is developed for a three-phase grid-connected inverter in a microgrid by using an adaptive population-based extremal optimization algorithm (APEO). Firstly, the optimal P-Q control issue of grid-connected inverters in a microgrid is formulated as a constrained optimization problem, where six parameters of three decoupled PI controllers are real-coded as the decision variables, and the integral time absolute error (ITAE) between the output and referenced active power and the ITAE between the output and referenced reactive power are weighted as the objective function. Then, an effective and efficient APEO algorithm with an adaptive mutation operation is proposed for solving this constrained optimization problem. The simulation and experiments for a 3kW three-phase grid-connected inverter under both nominal and variable reference active power values have shown that the proposed APEO-based P-Q control method outperforms the traditional Z-N empirical method, the adaptive genetic algorithm-based, and particle swarm optimization-based P-Q control methods.

Sign in / Sign up

Export Citation Format

Share Document