Automatic differentiation in prose

AbstractThe recently developed new algorithm for computing consistent initial values and Taylor coefficients for DAEs using projector-based constrained optimization opens new possibilities to apply Taylor series integration methods. In this paper, we show how corresponding projected explicit and implicit Taylor series methods can be adapted to DAEs of arbitrary index. Owing to our formulation as a projected optimization problem constrained by the derivative array, no explicit description of the inherent dynamics is necessary, and various Taylor integration schemes can be defined in a general framework. In particular, we address higher-order Padé methods that stand out due to their stability. We further discuss several aspects of our prototype implemented in Python using Automatic Differentiation. The methods have been successfully tested on examples arising from multibody systems simulation and a higher-index DAE benchmark arising from servo-constraint problems.

Download Full-text

Gumbel-softmax-based optimization: a simple general framework for optimization problems on graphs

Computational Social Networks ◽

10.1186/s40649-021-00086-z ◽

2021 ◽

Vol 8 (1) ◽

Author(s):

Yaoxin Li ◽

Jing Liu ◽

Guozheng Lin ◽

Yueyuan Hou ◽

Muyun Mou ◽

...

Keyword(s):

Objective Function ◽

Statistical Physics ◽

Automatic Differentiation ◽

Optimization Problems ◽

Vertex Cover ◽

Independent Set ◽

Computational Social Science ◽

Maximization Problem ◽

Maximum Independent Set ◽

Gradient Descent Algorithm

AbstractIn computer science, there exist a large number of optimization problems defined on graphs, that is to find a best node state configuration or a network structure, such that the designed objective function is optimized under some constraints. However, these problems are notorious for their hardness to solve, because most of them are NP-hard or NP-complete. Although traditional general methods such as simulated annealing (SA), genetic algorithms (GA), and so forth have been devised to these hard problems, their accuracy and time consumption are not satisfying in practice. In this work, we proposed a simple, fast, and general algorithm framework based on advanced automatic differentiation technique empowered by deep learning frameworks. By introducing Gumbel-softmax technique, we can optimize the objective function directly by gradient descent algorithm regardless of the discrete nature of variables. We also introduce evolution strategy to parallel version of our algorithm. We test our algorithm on four representative optimization problems on graph including modularity optimization from network science, Sherrington–Kirkpatrick (SK) model from statistical physics, maximum independent set (MIS) and minimum vertex cover (MVC) problem from combinatorial optimization on graph, and Influence Maximization problem from computational social science. High-quality solutions can be obtained with much less time-consuming compared to the traditional approaches.

Download Full-text