Algorithms for Convex Optimization

2021 ◽  
Author(s):  
Nisheeth K. Vishnoi
Keyword(s):  
Convex Optimization ◽  
Data Science ◽  
Optimization Problems ◽  
Algorithm Design ◽  
Continuous Optimization ◽  
Maximum Flow ◽  
Maximum Matching ◽  
Science Data ◽  
Mirror Descent ◽  
Continuous Optimization Problems

In the last few years, Algorithms for Convex Optimization have revolutionized algorithm design, both for discrete and continuous optimization problems. For problems like maximum flow, maximum matching, and submodular function minimization, the fastest algorithms involve essential methods such as gradient descent, mirror descent, interior point methods, and ellipsoid methods. The goal of this self-contained book is to enable researchers and professionals in computer science, data science, and machine learning to gain an in-depth understanding of these algorithms. The text emphasizes how to derive key algorithms for convex optimization from first principles and how to establish precise running time bounds. This modern text explains the success of these algorithms in problems of discrete optimization, as well as how these methods have significantly pushed the state of the art of convex optimization itself.

scholarly journals Decentralized and parallel primal and dual accelerated methods for stochastic convex programming problems

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Darina Dvinskikh ◽  
Alexander Gasnikov
Keyword(s):  
Convex Optimization ◽  
Inverse Problems ◽  
Convex Programming ◽  
Data Science ◽  
Optimization Problems ◽  
Stochastic Gradient ◽  
Logarithmic Factor ◽  
Accelerated Methods ◽  
Convex Optimization Problems ◽  
Primal And Dual

Abstract We introduce primal and dual stochastic gradient oracle methods for decentralized convex optimization problems. Both for primal and dual oracles, the proposed methods are optimal in terms of the number of communication steps. However, for all classes of the objective, the optimality in terms of the number of oracle calls per node takes place only up to a logarithmic factor and the notion of smoothness. By using mini-batching technique, we show that the proposed methods with stochastic oracle can be additionally parallelized at each node. The considered algorithms can be applied to many data science problems and inverse problems.

A Study of the Continuous Optimization Problem Using a Wood Robot Controlled by a Biologically Motivated System

2015 ◽  
Vol 137 (7) ◽  
Author(s):  
Jong-Chen Chen
Keyword(s):  
Physical Environment ◽  
Intelligent System ◽  
Optimization Problems ◽  
Learning Algorithm ◽  
Continuous Optimization ◽  
Physical Structure ◽  
Experimental Results ◽  
Self Organized ◽  
Selection Operators ◽  
Continuous Optimization Problems

Continuous optimization plays an increasingly significant role in everyday decision-making situations. Our group had previously developed a multilevel system called the artificial neuromolecular system (ANM) that possessed structure richness allowing variation and/or selection operators to act on it in order to generate a broad range of dynamic behaviors. In this paper, we used the ANM system to control the motions of a wooden walking robot named Miky. The robot was used to investigate the ANM system's capability to deal with continuous optimization problems through self-organized learning. Evolutionary learning algorithm was used to train the system and generate appropriate control. The experimental results showed that Miky was capable of learning in a continued manner in a physical environment. A further experiment was conducted by making some changes to Miky's physical structure in order to observe the system's capability to deal with the change. Detailed analysis of the experimental results showed that Miky responded to the change by appropriately adjusting its leg movements in space and time. The results showed that the ANM system possessed continuous optimization capability in coping with the change. Our findings from the empirical experiments might provide us another dimension of information of how to design an intelligent system comparatively friendlier than the traditional systems in assisting humans to walk.

scholarly journals A Particle Swarm Based Algorithm for Functional Distributed Constraint Optimization Problems

2020 ◽  
Vol 34 (05) ◽  
pp. 7111-7118
Author(s):  
Moumita Choudhury ◽  
Saaduddin Mahmud ◽  
Md. Mosaddek Khan
Keyword(s):  
Optimization Problems ◽  
State Of The Art ◽  
Particle Swarm ◽  
Continuous Optimization ◽  
Continuous Variables ◽  
Constraint Optimization ◽  
Solution Quality ◽  
Distributed Constraint Optimization ◽  
Continuous Optimization Problems ◽  
Constraint Optimization Problems

Distributed Constraint Optimization Problems (DCOPs) are a widely studied constraint handling framework. The objective of a DCOP algorithm is to optimize a global objective function that can be described as the aggregation of several distributed constraint cost functions. In a DCOP, each of these functions is defined by a set of discrete variables. However, in many applications, such as target tracking or sleep scheduling in sensor networks, continuous valued variables are more suited than the discrete ones. Considering this, Functional DCOPs (F-DCOPs) have been proposed that can explicitly model a problem containing continuous variables. Nevertheless, state-of-the-art F-DCOPs approaches experience onerous memory or computation overhead. To address this issue, we propose a new F-DCOP algorithm, namely Particle Swarm based F-DCOP (PFD), which is inspired by a meta-heuristic, Particle Swarm Optimization (PSO). Although it has been successfully applied to many continuous optimization problems, the potential of PSO has not been utilized in F-DCOPs. To be exact, PFD devises a distributed method of solution construction while significantly reducing the computation and memory requirements. Moreover, we theoretically prove that PFD is an anytime algorithm. Finally, our empirical results indicate that PFD outperforms the state-of-the-art approaches in terms of solution quality and computation overhead.

scholarly journals A Dynamic Adjusting Novel Global Harmony Search for Continuous Optimization Problems

Symmetry ◽  
2018 ◽  
Vol 10 (8) ◽  
pp. 337 ◽  
Author(s):  
Chui-Yu Chiu ◽  
Po-Chou Shih ◽  
Xuechao Li
Keyword(s):  
Optimization Problems ◽  
Harmony Search ◽  
Continuous Optimization ◽  
Genetic Mutation ◽  
Dynamic Adjustment ◽  
Mutation Probability ◽  
Adjustment Strategy ◽  
Improved Harmony Search ◽  
Searching Ability ◽  
Continuous Optimization Problems

A novel global harmony search (NGHS) algorithm, as proposed in 2010, is an improved algorithm that combines the harmony search (HS), particle swarm optimization (PSO), and a genetic algorithm (GA). Moreover, the fixed parameter of mutation probability was used in the NGHS algorithm. However, appropriate parameters can enhance the searching ability of a metaheuristic algorithm, and their importance has been described in many studies. Inspired by the adjustment strategy of the improved harmony search (IHS) algorithm, a dynamic adjusting novel global harmony search (DANGHS) algorithm, which combines NGHS and dynamic adjustment strategies for genetic mutation probability, is introduced in this paper. Moreover, extensive computational experiments and comparisons are carried out for 14 benchmark continuous optimization problems. The results show that the proposed DANGHS algorithm has better performance in comparison with other HS algorithms in most problems. In addition, the proposed algorithm is more efficient than previous methods. Finally, different strategies are suitable for different situations. Among these strategies, the most interesting and exciting strategy is the periodic dynamic adjustment strategy. For a specific problem, the periodic dynamic adjustment strategy could have better performance in comparison with other decreasing or increasing strategies. These results inspire us to further investigate this kind of periodic dynamic adjustment strategy in future experiments.

Sign in / Sign up

Export Citation Format

Share Document