Autonomous obstacle avoidance of UAV based on deep reinforcement learning1

In the intelligent unmanned systems, unmanned aerial vehicle (UAV) obstacle avoidance technology is the core and primary condition. Traditional algorithms are not suitable for obstacle avoidance in complex and changeable environments based on the limited sensors on UAVs. In this article, we use an end-to-end deep reinforcement learning (DRL) algorithm to achieve the UAV autonomously avoid obstacles. For the problem of slow convergence in DRL, a Multi-Branch (MB) network structure is proposed to ensure that the algorithm can get good performance in the early stage; for non-optimal decision-making problems caused by overestimation, the Revise Q-value (RQ) algorithm is proposed to ensure that the agent can choose the optimal strategy for obstacle avoidance. According to the flying characteristics of the rotor UAV, we build a V-Rep 3D physical simulation environment to test the obstacle avoidance performance. And experiments show that the improved algorithm can accelerate the convergence speed of agent and the average return of the round is increased by 25%.

Physical Model Test of Artificial Freezing-Inclined Shaft

Based on the vertical straight artificial freezing engineering in Northern Shaanxi, a three-dimensional (3D) physical simulation test system was developed, consisting of six parts, which are simulation box, shaft model, loading system, freezing system, external environment simulation system, and data acquisition system. The physical model and actual test results show that the 3D physical simulation test system is reasonable and reliable. The test model results show that the distance from the freezing pipe significantly affects the freezing wall temperature. For the case of four adjacent, two adjacent tangential freezing, and two adjacent axial freezing pipes, the cooling rates were 1.37, 2.79, and 1.96°C/h, respectively. The field measurement showed that the proximity to the freezing pipe increases the cooling rates. The cooling rates of points 1k#, 2k#, and 3k# were 25.61, 25.32, and 25.35 mm/d, respectively. The increment rates of vertical and horizontal freezing pressures with temperature were 8.78 and 2.97 kPa/°C, respectively. Furthermore, the freezing pressure time fitting formula was given. The calculated results of temperature and freezing pressure are consistent with the measured results, indicating the reasonability and reliability of the 3D physical simulation test scheme of the artificial freezing-inclined shaft in this work.

3D Physical Simulation Experiment of Edge Water Reservoir by Polymer/Surfactant Binary Flooding

To investigate the enhanced oil recovery (EOR) technology by chemical flooding in an edge water reservoir, a 3D physical simulation experimental device for the edge water reservoir was developed, and polymer/surfactant binary flooding experiments were carried out under different edge water energies. In addition, the effect and mechanism of binary flooding on EOR under different edge water energies were comprehensively analyzed. Experimental results show that edge water intrusion considerably affects EOR by binary flooding. The stronger the edge water energy, the worse the effect of EOR by binary flooding. Edge water possibly diluted the concentration of the chemical agent medium that is injected into the reservoir, and the degree of dilution varied in different regions. The dilution region was mainly distributed between the injection wells and edge water. The stronger the edge water energy, the higher the dilution multiple of chemical agent and the greater the recovery loss rate by binary flooding.