scholarly journals High-Level Path Planning for an Autonomous Sailboat Robot Using Q-Learning

Sensors ◽  
2020 ◽  
Vol 20 (6) ◽  
pp. 1550 ◽  
Author(s):  
Andouglas Gonçalves da Silva Silva Junior ◽  
Davi Henrique dos Santos ◽  
Alvaro Pinto Fernandes de Negreiros ◽  
João Moreno Vilas Boas de Souza Silva ◽  
Luiz Marcos Garcia Gonçalves
Keyword(s):  
Path Planning ◽  
Dead Zone ◽  
Weather Conditions ◽  
Target Point ◽  
Q Learning ◽  
Ground Station ◽  
Global Route ◽  
Global Planning ◽  
High Level ◽  
Path Planner

Path planning for sailboat robots is a challenging task particularly due to the kinematics and dynamics modelling of such kinds of wind propelled boats. The problem is divided into two layers. The first one is global were a general trajectory composed of waypoints is planned, which can be done automatically based on some variables such as weather conditions or defined by hand using some human–robot interface (a ground-station). In the second local layer, at execution time, the global route should be followed by making the sailboat proceed between each pair of consecutive waypoints. Our proposal in this paper is an algorithm for the global, path generation layer, which has been developed for the N-Boat (The Sailboat Robot project), in order to compute feasible sailing routes between a start and a target point while avoiding dangerous situations such as obstacles and borders. A reinforcement learning approach (Q-Learning) is used based on a reward matrix and a set of actions that changes according to wind directions to account for the dead zone, which is the region against the wind where the sailboat can not gain velocity. Our algorithm generates straight and zigzag paths accounting for wind direction. The path generated also guarantees the sailboat safety and robustness, enabling it to sail for long periods of time, depending only on the start and target points defined for this global planning. The result is the development of a complete path planner algorithm that, together with the local planner solved in previous work, can be used to allow the final developments of an N-Boat making it a fully autonomous sailboat.

scholarly journals Diversity Maintenance for Efficient Robot Path Planning

2020 ◽  
Vol 10 (5) ◽  
pp. 1721
Author(s):  
Petar Ćurković ◽  
Lovro Čehulić
Keyword(s):  
Path Planning ◽  
Autonomous Vehicles ◽  
Degrees Of Freedom ◽  
Population Based ◽  
Difficult Problem ◽  
Fitness Sharing ◽  
Diversity Maintenance ◽  
Novelty Search ◽  
High Level ◽  
Path Planner

Path planning is present in many areas, such as robotics, video games, and unmanned autonomous vehicles. In the case of robots, it is a primary low-level prerequisite for the successful execution of high-level tasks. It is a known and difficult problem to solve, especially in terms of finding optimal paths for robots working in complex environments. Recently, population-based methods for multi-objective optimization, i.e., swarm and evolutionary algorithms successfully perform on different path planning problems. Knowing the nature of the problem is hard for optimization algorithms, it is expected that population-based algorithms might benefit from some kind of diversity maintenance implementation. However, advantages and potential traps of implementing specific diversity maintenance methods into the evolutionary path planner have not been clearly spelled out and experimentally demonstrated. In this paper, we fill this gap and compare three diversity maintenance methods and their impact on the evolutionary planner for problems of different complexity. Crowding, fitness sharing, and novelty search are tailored to fit specific problems, implemented, and tested for two scenarios: mobile robot operating in a 2D maze, and 3 degrees of freedom (DOF) robot operating in a 3D environment including obstacles. Results indicate that the novelty search outperforms the other two methods for problem domains of higher complexity.

scholarly journals Rollover-Free Path Planning for Off-Road Autonomous Driving

Electronics ◽  
2019 ◽  
Vol 8 (6) ◽  
pp. 614
Author(s):  
Xingyu Li ◽  
Bo Tang ◽  
John Ball ◽  
Matthew Doude ◽  
Daniel W. Carruth
Keyword(s):  
Path Planning ◽  
Free Path ◽  
Autonomous Driving ◽  
Planning Method ◽  
Local Planning ◽  
Planning Stage ◽  
Local Path Planning ◽  
Global Route ◽  
Local Path ◽  
Global Planning

Perception, planning, and control are three enabling technologies to achieve autonomy in autonomous driving. In particular, planning provides vehicles with a safe and collision-free path towards their destinations, accounting for vehicle dynamics, maneuvering capabilities in the presence of obstacles, traffic rules, and road boundaries. Existing path planning algorithms can be divided into two stages: global planning and local planning. In the global planning stage, global routes and the vehicle states are determined from a digital map and the localization system. In the local planning stage, a local path can be achieved based on a global route and surrounding information obtained from sensors such as cameras and LiDARs. In this paper, we present a new local path planning method, which incorporates a vehicle’s time-to-rollover model for off-road autonomous driving on different road profiles for a given predefined global route. The proposed local path planning algorithm uses a 3D occupancy grid and generates a series of 3D path candidates in the s-p coordinate system. The optimal path is then selected considering the total cost of safety, including obstacle avoidance, vehicle rollover prevention, and comfortability in terms of path smoothness and continuity with road unevenness. The simulation results demonstrate the effectiveness of the proposed path planning method for various types of roads, indicating its wide practical applications to off-road autonomous driving.

scholarly journals A 2D Optimal Path Planning Algorithm for Autonomous Underwater Vehicle Driving in Unknown Underwater Canyons

2021 ◽  
Vol 9 (3) ◽  
pp. 252
Author(s):  
Yushan Sun ◽  
Xiaokun Luo ◽  
Xiangrui Ran ◽  
Guocheng Zhang
Keyword(s):  
Path Planning ◽  
Obstacle Avoidance ◽  
Autonomous Underwater Vehicles ◽  
Optimal Path ◽  
Small Scale ◽  
Target Point ◽  
Safe Driving ◽  
Policy Gradient ◽  
Planning Algorithm ◽  
Path Planning Algorithm

This research aims to solve the safe navigation problem of autonomous underwater vehicles (AUVs) in deep ocean, which is a complex and changeable environment with various mountains. When an AUV reaches the deep sea navigation, it encounters many underwater canyons, and the hard valley walls threaten its safety seriously. To solve the problem on the safe driving of AUV in underwater canyons and address the potential of AUV autonomous obstacle avoidance in uncertain environments, an improved AUV path planning algorithm based on the deep deterministic policy gradient (DDPG) algorithm is proposed in this work. This method refers to an end-to-end path planning algorithm that optimizes the strategy directly. It takes sensor information as input and driving speed and yaw angle as outputs. The path planning algorithm can reach the predetermined target point while avoiding large-scale static obstacles, such as valley walls in the simulated underwater canyon environment, as well as sudden small-scale dynamic obstacles, such as marine life and other vehicles. In addition, this research aims at the multi-objective structure of the obstacle avoidance of path planning, modularized reward function design, and combined artificial potential field method to set continuous rewards. This research also proposes a new algorithm called deep SumTree-deterministic policy gradient algorithm (SumTree-DDPG), which improves the random storage and extraction strategy of DDPG algorithm experience samples. According to the importance of the experience samples, the samples are classified and stored in combination with the SumTree structure, high-quality samples are extracted continuously, and SumTree-DDPG algorithm finally improves the speed of the convergence model. Finally, this research uses Python language to write an underwater canyon simulation environment and builds a deep reinforcement learning simulation platform on a high-performance computer to conduct simulation learning training for AUV. Data simulation verified that the proposed path planning method can guide the under-actuated underwater robot to navigate to the target without colliding with any obstacles. In comparison with the DDPG algorithm, the stability, training’s total reward, and robustness of the improved Sumtree-DDPG algorithm planner in this study are better.

scholarly journals Collision-free path planning for welding manipulator via hybrid algorithm of deep reinforcement learning and inverse kinematics

2021 ◽  
Author(s):  
Jie Zhong ◽  
Tao Wang ◽  
Lianglun Cheng
Keyword(s):  
Reinforcement Learning ◽  
Path Planning ◽  
Free Path ◽  
Inverse Kinematics ◽  
Multiple Dimensions ◽  
Continuous State ◽  
Planning Algorithm ◽  
Convergence Performance ◽  
Path Planner ◽  
Action Spaces

AbstractIn actual welding scenarios, an effective path planner is needed to find a collision-free path in the configuration space for the welding manipulator with obstacles around. However, as a state-of-the-art method, the sampling-based planner only satisfies the probability completeness and its computational complexity is sensitive with state dimension. In this paper, we propose a path planner for welding manipulators based on deep reinforcement learning for solving path planning problems in high-dimensional continuous state and action spaces. Compared with the sampling-based method, it is more robust and is less sensitive with state dimension. In detail, to improve the learning efficiency, we introduce the inverse kinematics module to provide prior knowledge while a gain module is also designed to avoid the local optimal policy, we integrate them into the training algorithm. To evaluate our proposed planning algorithm in multiple dimensions, we conducted multiple sets of path planning experiments for welding manipulators. The results show that our method not only improves the convergence performance but also is superior in terms of optimality and robustness of planning compared with most other planning algorithms.

scholarly journals An Improved Q-learning Algorithm for Path-Planning of a Mobile Robot

2012 ◽  
Vol 51 (9) ◽  
pp. 40-46 ◽  
Author(s):  
Pradipta KDas ◽  
S. C. Mandhata ◽  
H. S. Behera ◽  
S. N. Patro
Keyword(s):  
Path Planning ◽  
Mobile Robot ◽  
Learning Algorithm ◽  
Q Learning

Path Planning for Spheres in Three Dimensional Environments With Low Interference Index

1994 ◽  
Author(s):  
Duane W. Storti ◽  
Debasish Dutta
Keyword(s):  
Path Planning ◽  
Free Path ◽  
Configuration Space ◽  
Local Knowledge ◽  
Three Dimensional ◽  
Target Point ◽  
Planning Problem ◽  
Spherical Object ◽  
Starting Point ◽  
Path Planning Problem

Abstract We consider the path planning problem for a spherical object moving through a three-dimensional environment composed of spherical obstacles. Given a starting point and a terminal or target point, we wish to determine a collision free path from start to target for the moving sphere. We define an interference index to count the number of configuration space obstacles whose surfaces interfere simultaneously. In this paper, we present algorithms for navigating the sphere when the interference index is ≤ 2. While a global calculation is necessary to characterize the environment as a whole, only local knowledge is needed for path construction.

Optimal path planning of a disinfection mobile robot against COVID-19 in a ROS-based research platform

2021 ◽  
Vol 16 (4) ◽  
pp. 405-417
Author(s):  
L. Banjanovic-Mehmedovic ◽  
I. Karabegovic ◽  
J. Jahic ◽  
M. Omercic
Keyword(s):  
Path Planning ◽  
Optimal Path ◽  
Pso Algorithm ◽  
Optimal Path Planning ◽  
Localization And Mapping ◽  
Increasing Demand ◽  
Window Approach ◽  
Path Planner ◽  
Hardware Platforms ◽  
Reactive Collision

Due to COVID-19 pandemic, there is an increasing demand for mobile robots to substitute human in disinfection tasks. New generations of disinfection robots could be developed to navigate in high-risk, high-touch areas. Public spaces, such as airports, schools, malls, hospitals, workplaces and factories could benefit from robotic disinfection in terms of task accuracy, cost, and execution time. The aim of this work is to integrate and analyse the performance of Particle Swarm Optimization (PSO) algorithm, as global path planner, coupled with Dynamic Window Approach (DWA) for reactive collision avoidance using a ROS-based software prototyping tool. This paper introduces our solution – a SLAM (Simultaneous Localization and Mapping) and optimal path planning-based approach for performing autonomous indoor disinfection work. This ROS-based solution could be easily transferred to different hardware platforms to substitute human to conduct disinfection work in different real contaminated environments.

SENSITIVITY OF FORSYTHIA POLLEN TO THE FACTORS OF THE ENVIRONMENT OF THE MEGAPOLIS

2021 ◽  
pp. 56-74
Author(s):  
Matiashuk R. ◽  
Tkachenko I.
Keyword(s):  
Air Pollution ◽  
Ecological Status ◽  
Pollution Index ◽  
Weather Conditions ◽  
Botanical Garden ◽  
Control Value ◽  
Adverse Weather ◽  
Long Time ◽  
Growing Conditions ◽  
High Level

The sensitivity of the reproductive structures of Forsythiasuspensato the complex influence of undifferentiated environmental factors has been studied.The monitoring sites are located in 15 different park ecosystems in 6 administrative districts of Kyiv. Data from the Borys Sreznevsky Central Geophysical Observatory (air pollution index (API) and meteorological conditions for 2018-2020) were used to assess the conditions of the growthenvironment. The influence of a complex of ecologically important factors during the flowering offorsythia on the quality of the formedpollen is noted. Thus, growing plantsfor a long time in conditions with a low level of air pollution (APIup to 5.0) in abnormal weather conditions in 2020,40-50% less fertile grainswere formed. And forplants, which grow in areas with increasedlevel (API5.0-7.0) and high level (API7.0-14.0) of air pollution, the share of fertile grains in the pollen population decreased by 60-80%. In the closed bud,the pollen has higher resistance to a complex of exogenous growth factors. Forsythia plants, which are located in large parklands, lose less pollen quality in adverse weather conditions and affected byurbotechnogenic factors. F. suspensa is an acceptable indicator of the level of environmental pollution by the deviation of pollen fertility from the control value. In areas with high aerogenic load, for example, areas with large highways (Bus Station «Darnytsia»), as well as with a significant recreational load (HolosiivskyiPark, Recreation Park on the Olena Teliha Street) much smaller pollen is formed. Itis noted that the conditions of forsythia growth affect the quantitative indicators of the formed pollen not only in the flower but also in the closed bud, which confirms the chronic effect of the complex of ingredients of aerotechnogenic emissions on plants of this species. The coefficient of sterility of pollen (CS) was used to objectively compare the data of 2019 and 2020 on the condition of the generative organs of F. suspensa in the studied areas. The calculation of the CS confirmed that in the closedbud pollen has a higher resistance to exogenous factors. In areas with high and increased levels of air pollution,during budding and flowering of plants (March-May,) there is a much higher CS of pollen of F. suspensа. Significant parkland territoriesof the city (for example, HolosiivskyiPark, Botanical Garden named after O. V. Fomin) provide less stressful conditions for growth and development of plants, even with the "very high" level of air pollution (ISA above 14.0) observed in April 2020 on this territory.The study of the susceptibility of F. suspensаpollen to growing conditions will be continued, as the prospects of using this species for bioindication of ecological status and zoning of park ecosystems of Kyiv according to the gradient of anthropogenic impact have been revealed.Key words:forsythia, fertility, coefficient of sterility of pollen, bioindication. Проведене дослідження чутливості репродуктивних структур Forsythiasuspensaдо комплексного впливу недиференційованих факторів навколишнього середовища. Моніторингові ділянки розташовані в 15 різних паркових екосистемах 6 адміністративних районівКиєва. Для оцінки умов середовища вирощування рослин використані дані Центральної геофізичної обсерваторії імені Бориса Срезневського (індекс забруднення атмосферного повітря (ІЗА) та метеорологічні умови за 2018-2020 рр.). Відмічено вплив комплексу екологічно важливих факторів (за показниками відхилення від норми середньої місячної температури повітря та місячної кількості опадів у Києві) уперіод квітування форзиції на якість сформованого пилку. Так, за тривалої експозиції рослин в умовах з низьким рівнем забруднення повітря (ІЗА до 5,0) в аномальних погодних умовах 2020 р. сформувалось на 40-50% менше фертильних зерен. А у форзиції, яка росте на територіях з підвищеним (ІЗА 5,0-7,0) та високим (ІЗА 7,0-14,0) рівнями забруднення частка фертильних зерен в популяції пилку зменшилась на 60-80%. У закритому бутоні пилок має вищу стійкість до комплексу екзогенних факторів середовища зростання. Рослини форзиції, які розташовані у значних за площею паркових насадженнях, менше втрачають якість пилку за несприятливих погодних умов та дії урботехногенних чинників. За відхиленням показника фертильності пилку від контрольного значення F. suspensaє допустимим індикатором рівня забруднення середовища. На територіях з підвищеним аерогенним навантаженням, наприклад, ділянки з автотранспортними магістралями (Автостанція «Дарниця»), а також зі значним рекреаційним навантаженням (Голосіївський парк імені М. Рильського, Парк відпочинку по вул. Олени Теліги)формується значно дрібніший пилок. Відмічено, що умови росту позначаються на кількісних показниках сформованого пилку не лише в квітці, але й в закритому бутоні, що підтверджує хронічний вплив комплексу інгредієнтів аеротехногенних викидів нарослини цього виду. Для об’єктивного співставлення даних 2019 і 2020 рр. щодо стану генеративних органів F. suspensaна досліджених ділянках був використаний коефіцієнт стерильності (КС) пилку. Розрахунок КС підтвердив, що в закритому бутоні пилок має вищу стійкість до впливу екзогенних чинників. Вищий КС був у F. suspense, з ділянок, на яких в період бутонізації і квітування рослин (березень-травень) відмічений високий і підвищений рівень забруднення атмосфери. Значні за площею паркові насадження міста (наприклад, Голосіївський парк, Ботанічний сад ім.акад. О.В. Фоміна) забезпечують менш напружені умови росту і розвитку рослин навіть при відміченому в квітні 2020 р. «дуже високому» рівні забруднення повітря (ІЗА вище 14,0) на цих територіях. Дослідження чутливості пилку F. suspenseдо умов вирощування буде продовжене,оскільки виявлена перспективність використання цього виду для біоіндикації екологічного стану та зонування паркових екосистем Києва за градієнтом антропогенного впливу.Ключові слова: форзиція, фертильність, індекс стерильності, біоіндикація.

Sign in / Sign up

Export Citation Format

Share Document