Data Efficient Reinforcement Learning for Integrated Lateral Planning and Control in Automated Parking System

Reinforcement learning (RL) is a promising direction in automated parking systems (APSs), as integrating planning and tracking control using RL can potentially maximize the overall performance. However, commonly used model-free RL requires many interactions to achieve acceptable performance, and model-based RL in APS cannot continuously learn. In this paper, a data-efficient RL method is constructed to learn from data by use of a model-based method. The proposed method uses a truncated Monte Carlo tree search to evaluate parking states and select moves. Two artificial neural networks are trained to provide the search probability of each tree branch and the final reward for each state using self-trained data. The data efficiency is enhanced by weighting exploration with parking trajectory returns, an adaptive exploration scheme, and experience augmentation with imaginary rollouts. Without human demonstrations, a novel training pipeline is also used to train the initial action guidance network and the state value network. Compared with path planning and path-following methods, the proposed integrated method can flexibly co-ordinate the longitudinal and lateral motion to park a smaller parking space in one maneuver. Its adaptability to changes in the vehicle model is verified by joint Carsim and MATLAB simulation, demonstrating that the algorithm converges within a few iterations. Finally, experiments using a real vehicle platform are used to further verify the effectiveness of the proposed method. Compared with obtaining rewards using simulation, the proposed method achieves a better final parking attitude and success rate.

Bifurcation analysis of a vibro-impact experimental rig with two-sided constraint

In this paper we carry out an in-depth experimental and numerical investigation of a vibro-impact rig with a two-sided constraint and an external excitation given by a rectangular waveform. The rig, presenting forward and backward drifts, consists of an inner vibrating shaft intermittently impacting with its holding frame. Our interests focus on the multistability and the bifurcation structure observed in the system under two different contacting surfaces. For this purpose, we propose a mathematical model describing the rig dynamics and perform a detailed bifurcation analysis via path-following methods for nonsmooth dynamical systems, using the continuation platform COCO. Our study shows that multistability is produced by the interplay between two fold bifurcations, which give rise to hysteresis in the system. The investigation also reveals the presence of period-doubling bifurcations of limit cycles, which in turn are responsible for the creation of period-2 solutions for which the rig reverses its direction of progression. Furthermore, our study considers a two-parameter bifurcation analysis focusing on directional control, using the period of external excitation and the duty cycle of the rectangular waveform as the main control parameters.

Path-Following Methods for Calculating Linear Surface Wave Dispersion Relations on Vertical Shear Flows

The path-following scheme in Loisel and Maxwell (SIAM J Matrix Anal Appl 39(4):1726–1749, 2018) is adapted to efficiently calculate the dispersion relation curve for linear surface waves on an arbitrary vertical shear current. This is equivalent to solving the Rayleigh stability equation with linearized free-surface boundary condition for each sought point on the curve. Taking advantage of the analyticity of the dispersion relation, a path-following or continuation approach is adopted. The problem is discretized using a collocation scheme, parametrized along either a radial or angular path in the wave vector plane, and differentiated to yield a system of ODEs. After an initial eigenproblem solve using QZ decomposition, numerical integration proceeds along the curve using linear solves as the Runge–Kutta $$F(\cdot )$$ F ( · ) function; thus, many QZ decompositions on a size 2N companion matrix are exchanged for one QZ decomposition and a small number of linear solves on a size N matrix. A piecewise interpolant provides dense output. The integration represents a nominal setup cost whereafter very many points can be computed at negligible cost whilst preserving high accuracy. Furthermore, a two-dimensional interpolant suitable for scattered data query points in the wave vector plane is described. Finally, a comparison is made with existing numerical methods for this problem, revealing that the path-following scheme is the most competitive algorithm for this problem whenever calculating more than circa 1,000 data points or relative normwise accuracy better than $$10^{-4}$$ 10 - 4 is sought.

Convex Optimization and Numerical Issues

This chapter aims at providing the intuition behind convex optimization algorithms and addresses their effective use with floating-point implementation. It first briefly presents the algorithms, assuming a real semantics. As outlined in Chapter 4, convex conic programming is supported by different methods depending on the cone considered. The most known approach for linear constraints is the simplex method by Dantzig. While having an exponential-time complexity with respect to the number of constraints, the simplex method performs well in general. Another method is the set of interior point methods, initially proposed by Karmarkar and made popular by Nesterov and Nemirovski. They can be characterized as path-following methods in which a sequence of local linear problems are solved, typically by Newton's method. After these algorithms are considered, the chapter discusses approaches to obtain sound results.

Trusses Nonlinear Problems Solution with Numerical Methods of Cubic Convergence Order

A large part of the numerical procedures for obtaining the equilibrium path or load-displacement curve of structural problems with nonlinear behavior is based on the Newton-Raphson iterative scheme, to which is coupled the path-following methods. This paper presents new algorithms based on Potra-Pták, Chebyshev and super-Halley methods combined with the Linear Arc-Length path-following method. The main motivation for using these methods is the cubic order convergence. To elucidate the potential of our approach, we present an analysis of space and plane trusses problems with geometric nonlinearity found in the literature. In this direction, we will make use of the Positional Finite Element Method, which considers the nodal coordinates as variables of the nonlinear system instead of displacements. The numerical results of the simulations show the capacity of the computational algorithm developed to obtain the equilibrium path with force and displacement limits points. The implemented iterative methods exhibit better efficiency as the number of time steps and necessary accumulated iterations until convergence and processing time, in comparison with classic methods of Newton-Raphson and Modified Newton-Raphson.

Path-Following Bifurcation Analysis of Church Bell Dynamics

In this paper, we perform a path-following bifurcation analysis of church bell to gain an insight into the governing dynamics of the yoke–bell–clapper system. We use an experimentally validated hybrid dynamical model based on the detailed measurements of a real church bell. Numerical analysis is performed both by a direct numerical integration and a path-following methods using a new numerical toolbox ABESPOL (Chong, 2016, “Numerical Modeling and Stability Analysis of Non-Smooth Dynamical Systems Via ABESPOL,” Ph.D. thesis, University of Aberdeen, Aberdeen, UK) based on COCO (Dankowicz and Schilder, Recipes for Continuation (Computational Science and Engineering), Society for Industrial and Applied Mathematics, Philadelphia, PA). We constructed one-parameter diagrams that allow to characterize the most common dynamical states and to investigate the mechanisms of their dynamic stability. A novel method allowing to locate the regions in the parameters' space ensuring robustness of bells' effective performance is presented.