Deep Reinforcement Learning for Vectored Thruster Autonomous Underwater Vehicle Control

Autonomous underwater vehicles (AUVs) are widely used to accomplish various missions in the complex marine environment; the design of a control system for AUVs is particularly difficult due to the high nonlinearity, variations in hydrodynamic coefficients, and external force from ocean currents. In this paper, we propose a controller based on deep reinforcement learning (DRL) in a simulation environment for studying the control performance of the vectored thruster AUV. RL is an important method of artificial intelligence that can learn behavior through trial-and-error interactions with the environment, so it does not need to provide an accurate AUV control model that is very hard to establish. The proposed RL algorithm only uses the information that can be measured by sensors inside the AUVs as the input parameters, and the outputs of the designed controller are the continuous control actions, which are the commands that are set to the vectored thruster. Moreover, a reward function is developed for deep RL controller considering different factors which actually affect the control accuracy of AUV navigation control. To confirm the algorithm’s effectiveness, a series of simulations are carried out in the designed simulation environment, which is a method to save time and improve efficiency. Simulation results prove the feasibility of the deep RL algorithm applied to the control system for AUV. Furthermore, our work also provides an optional method for robot control problems to deal with improving technology requirements and complicated application environments.

Experiments on high-performance maneuvers control for a work-class 3000-m remote operated vehicle

The challenge of high-performance maneuvers for a deep-ocean underwater vehicle is to design proper controllers and ensure the high accuracy of vehicle state measurements. Current underwater vehicle control solutions are considered to be a unique controller designed to reject the disturbance and noise. However, the controller is based on a precise hydrodynamic model. It requires multiple underwater experiments and complex theoretical analysis. In this article, a hybrid control strategy is presented for the work-class remote operated vehicle. It tries to compose several proportion–integral–differential controllers into one intelligent sequence in terms of task requirements. The proposed approach employs the iterative learning method to improve the performance of the depth holding and the way-point tracking. A remote operated vehicle system weighted about 1.5 tons is provided as a physical platform for scientific investigations using acoustic Doppler current profiler, inertial navigation system, depth sensor, and an altimeter. Results from the simulation and the experiment in the basin show that the proposed approach provides high accuracy at both conditions: tracking way-points with or without ocean currents and disturbances, which show the effectiveness of the proposed approach. In addition, the thrust vector defined for a propeller facilitates the control of underwater vehicle as the thruster configuration changes.

The Application of PSO-AFSA Method in Parameter Optimization for Underactuated Autonomous Underwater Vehicle Control

In consideration of the difficulty in determining the parameters of underactuated autonomous underwater vehicles in multi-degree-of-freedom motion control, a hybrid method that combines particle swarm optimization (PSO) with artificial fish school algorithm (AFSA) is proposed in this paper. The optimization process of the PSO-AFSA method is firstly introduced. With the control simulation models in the horizontal plane and vertical plane, the PSO-AFSA method is elaborated when applied in control parameter optimization for an underactuated autonomous underwater vehicle. Both simulation tests and field trials were carried out to prove the efficiency of the PSO-AFSA method in underactuated autonomous underwater vehicle control parameter optimization. The optimized control parameters showed admirable control quality by enabling the underactuated autonomous underwater vehicle to reach the desired states with fast convergence.