scholarly journals UAV Motion Strategies in Uncertain Dynamic Environments: A Path Planning Method Based on Q-Learning Strategy

2018 ◽  
Vol 8 (11) ◽  
pp. 2169 ◽  
Author(s):  
Jun-hui Cui ◽  
Rui-xuan Wei ◽  
Zong-cheng Liu ◽  
Kai Zhou
Keyword(s):  
Path Planning ◽  
Cost Function ◽  
Tracking Error ◽  
Control Variable ◽  
Dynamic Environments ◽  
Tracking Accuracy ◽  
Bank Angle ◽  
Action Sequence ◽  
Q Learning ◽  
The Cost

A solution framework for UAV motion strategies in uncertain dynamic environments is constructed in this paper. Considering that the motion states of UAV might be influenced by some dynamic uncertainties, such as control strategies, flight environments, and any other bursting-out threats, we model the uncertain factors that might cause such influences to the path planning of the UAV, unified as an unobservable part of the system and take the acceleration together with the bank angle of the UAV as a control variable. Meanwhile, the cost function is chosen based on the tracking error, then the control instructions and flight path for UAV can be achieved. Then, the cost function can be optimized through Q-learning, and the best UAV action sequence for conflict avoidance under the moving threat environment can be obtained. According to Bellman’s optimization principle, the optimal action strategies can be obtained from the current confidence level. The method in this paper is more in line with the actual UAV path planning, since the generation of the path planning strategy at each moment takes into account the influence of the UAV control strategy on its motion at the next moment. The simulation results show that all the planning paths that are created according to the solution framework proposed in this paper have a very high tracking accuracy, and this method has a much shorter processing time as well as a shorter path it can create.

scholarly journals Total Harmonic Distortion Oriented Finite Control Set Model Predictive Control for Single-Phase Inverters

Energies ◽  
2018 ◽  
Vol 11 (12) ◽  
pp. 3467 ◽  
Author(s):  
Po Li ◽  
Ruiyu Li ◽  
Haifeng Feng
Keyword(s):  
Model Predictive Control ◽  
Cost Function ◽  
Predictive Control ◽  
Single Phase ◽  
Tracking Error ◽  
Harmonic Distortion ◽  
Total Harmonic Distortion ◽  
Finite Control ◽  
Control Set ◽  
The Cost

Inverters are commonly controlled to generate AC current and Total Harmonic Distortion (THD) is the core index in judging the control effect. In this paper, a THD oriented Finite Control Set Model Predictive Control (FCS MPC) scheme is proposed for the single-phase inverter, where a optimization problem is solved to obtain the switching law for realization. Different from the traditional cost function, which focuses on the instantaneous deviation of amplitude between predictive current and its reference, we redesign a cost function that is the linear combination of the current fundamental tracking error, instantaneous THD value and DC component in one fundamental cycle (for 50 Hz, it is 0.02 s). Iterative method is developed for rapid calculation of this cost function. By choosing a switching state from a FCS to minimize the cost function, a FCS MPC is finally constructed. Simulation results in Matlab/Simulink and experimental results on rapid control prototype platform show the effect of this method. Analyses illustrate that, by choosing suitable weight of the cost function, the performance of this THD oriented FCS MPC method is better than the traditional one.

scholarly journals A Counterexample to Temporal Differences Learning

1995 ◽  
Vol 7 (2) ◽  
pp. 270-279 ◽  
Author(s):  
Dimitri P. Bertsekas
Keyword(s):  
Neural Network ◽  
Markov Chain ◽  
Cost Function ◽  
Least Squares ◽  
Temporal Differences ◽  
Q Learning ◽  
Absorbing Markov Chain ◽  
Transition Costs ◽  
Compact Representations ◽  
The Cost

Sutton's TD(λ) method aims to provide a representation of the cost function in an absorbing Markov chain with transition costs. A simple example is given where the representation obtained depends on λ. For λ = 1 the representation is optimal with respect to a least-squares error criterion, but as λ decreases toward 0 the representation becomes progressively worse and, in some cases, very poor. The example suggests a need to understand better the circumstances under which TD(0) and Q-learning obtain satisfactory neural network-based compact representations of the cost function. A variation of TD(0) is also given, which performs better on the example.

scholarly journals Least Trimmed Squares Approach to Lucas-Kanade Algorithm in Object Tracking Problems

2013 ◽  
Vol 2013 ◽  
pp. 1-6
Author(s):  
Yih-Lon Lin
Keyword(s):  
Cost Function ◽  
Object Tracking ◽  
Vehicle Tracking ◽  
Tracking Accuracy ◽  
Least Trimmed Squares ◽  
Environment Adaptation ◽  
Tracking Process ◽  
Important Research Topic ◽  
Process Simulations ◽  
The Cost

The object tracking problem is an important research topic in computer vision. For real applications such as vehicle tracking and face tracking, there are many efficient and real-time algorithms. In this study, we will focus on the Lucas-Kanade (LK) algorithm for object tracking. Although this method is time consuming, it is effective in tracking accuracy and environment adaptation. In the standard LK method, the sum of squared errors is used as the cost function, while least trimmed squares is adopted as the cost function in this study. The resulting estimator is robust against outliers caused by noises and occlusions in the tracking process. Simulations are provided to show that the proposed algorithm outperforms the standard LK method in the sense that it is robust against the outliers in the object tracking problems.

scholarly journals D* lite algorithm based path planning of mobile robot in static Environment

2012 ◽  
pp. 286-290
Author(s):  
Pradipta kumar Das ◽  
S .N. Patro ◽  
C. N. Panda ◽  
Bunil Balabantaray
Keyword(s):  
Path Planning ◽  
Mobile Robot ◽  
Cost Function ◽  
Shortest Path ◽  
Target Position ◽  
Initial Position ◽  
Distance Metric ◽  
Unknown Environment ◽  
Feasible Path ◽  
The Cost

In this paper, we study the path planning for khepera II mobile robot in an unknown environment. The well known heuristic D* lite algorithm is implemented to make the mobile robot navigate through static obstacles and find the shortest path from an initial position to a target position by avoiding the obstacles. and to perform efficient re-planning during exploration. The proposed path finding strategy is designed in a grid-map form of an unknown environment with static unknown obstacles. The robot moves within the unknown environment by sensing and avoiding the obstacles coming across its way towards the target. When the mission is executed, it is necessary to plan an optimal or feasible path for itself avoiding obstructions in its way and minimizing a cost such as time, energy, and distance. In our study we have considered the distance metric as the cost function.

scholarly journals Cost-Reference Particle Filter for Cognitive Radar Tracking Systems with Unknown Statistics

Sensors ◽  
2020 ◽  
Vol 20 (13) ◽  
pp. 3669 ◽  
Author(s):  
Lei Zhong ◽  
Yong Li ◽  
Wei Cheng ◽  
Yi Zheng
Keyword(s):  
Cost Function ◽  
Particle Filter ◽  
Particle Filtering ◽  
Estimation Error ◽  
Cognitive Structure ◽  
Tracking Accuracy ◽  
Cognitive Radar ◽  
Radar Tracking ◽  
Function Design ◽  
The Cost

A novel robust particle filtering algorithm is proposed for updating both the waveform and noise parameter for tracking accuracy simultaneously and adaptively. The approach is a significant step for cognitive radar towards more robust tracking in random dynamic systems with unknown statistics. Meanwhile, as an intelligent sensor, it would be most desirable for cognitive radar to develop the application of a traditional filter to be adaptive and to expand the adaptation to a wider scope. In this paper, after analysis of the Bayesian bounds and the corresponding cost function design, we propose the cognitive radar tracking method based on a particle filter by completely reconstructing the propagation and the update process with a cognitive structure. Moreover, we develop the cost-reference particle filter based on optimizing the cost function design according to the complicated system or environment with unknown statistics. With this method, the update of the estimation cost and variance arrives at the approximate optimization, and the estimation error can be more adjacent to corresponding low bounds. Simulations about the tracking implementation in unknown noise are utilized to demonstrate the superiority of the proposed algorithm to the existing methods in traditional radar.

Optimal cooperative path planning of unmanned aerial vehicles by a parallel genetic algorithm

Robotica ◽  
2014 ◽  
Vol 34 (4) ◽  
pp. 823-836 ◽  
Author(s):  
Hamed Shorakaei ◽  
Mojtaba Vahdani ◽  
Babak Imani ◽  
Ali. Gholami
Keyword(s):  
Genetic Algorithm ◽  
Path Planning ◽  
Cost Function ◽  
Simple Formula ◽  
Parallel Genetic Algorithm ◽  
Probability Map ◽  
Genetic Encoding ◽  
Cell Position ◽  
The Cost ◽  
Unique Vector

SUMMARYThe current paper presents a path planning method based on probability maps and uses a new genetic algorithm for a group of UAVs. The probability map consists of cells that display the probability which the UAV will not encounter a hostile threat. The probability map is defined by three events. The obstacles are modeled in the probability map, as well. The cost function is defined such that all cells are surveyed in the path track. The simple formula based on the unique vector is presented to find this cell position. Generally, the cost function is formed by two parts; one part for optimizing the path of each UAV and the other for preventing UAVs from collision. The first part is a combination of safety and length of path and the second part is formed by an exponential function. Then, the optimal paths of each UAV are obtained by the genetic algorithm in a parallel form. According to the dimensions of path planning, genetic encoding has two or three indices. A new genetic operator is introduced to select an appropriate pair of chromosome for crossover operation. The effectiveness of the method is shown by several simulations.

scholarly journals An Overlooked Issue of Variational Data Assimilation

2015 ◽  
Vol 143 (10) ◽  
pp. 3925-3930 ◽  
Author(s):  
Benjamin Ménétrier ◽  
Thomas Auligné
Keyword(s):  
Data Assimilation ◽  
Cost Function ◽  
Control Variable ◽  
Variational Data Assimilation ◽  
Inner Product ◽  
Background Error ◽  
Error Covariance ◽  
Definition Of ◽  
Linear Unbiased Estimate ◽  
The Cost

Abstract The control variable transform (CVT) is a keystone of variational data assimilation. In publications using such a technique, the background term of the transformed cost function is defined as a canonical inner product of the transformed control variable with itself. However, it is shown in this paper that this practical definition of the cost function is not correct if the CVT uses a square root of the background error covariance matrix that is not square. Fortunately, it is then shown that there is a manifold of the control space for which this flaw has no impact, and that most minimizers used in practice precisely work in this manifold. It is also shown that both correct and practical transformed cost functions have the same minimum. This explains more rigorously why the CVT is working in practice. The case of a singular is finally detailed, showing that the practical cost function still reaches the best linear unbiased estimate (BLUE).

scholarly journals Path Planning of Mobile Robot in Unknown Environment

2010 ◽  
pp. 122-127
Author(s):  
Pradipta kumar Das ◽  
Romesh Laishram ◽  
Amit Konar
Keyword(s):  
Path Planning ◽  
Mobile Robot ◽  
Cost Function ◽  
Shortest Path ◽  
Target Position ◽  
Initial Position ◽  
A Algorithm ◽  
Unknown Environment ◽  
Feasible Path ◽  
The Cost

In this paper, we study the online path planning for khepera II mobile robot in an unknown environment. The well known heuristic A* algorithm is implemented to make the mobile robot navigate through static obstacles and find the shortest path from an initial position to a target position by avoiding the obstacles. The proposed path finding strategy is designed in a grid-map form of an unknown environment with static unknown obstacles. When the mission is executed, it is necessary to plan an optimal or feasible path for itself avoiding obstructions in its way and minimizing a cost such as time, energy, and distance. In our study we have considered the distance and time metric as the cost function.

Sign in / Sign up

Export Citation Format

Share Document