Approximating Markov Chain Approach to Optimal Feedback Control of a Flexible Needle

2016 ◽  
Vol 138 (11) ◽  
Author(s):  
Javad Sovizi ◽  
Suren Kumar ◽  
Venkat Krovi
Keyword(s):  
Markov Chain ◽  
Feedback Control ◽  
Stochastic Systems ◽  
Control Policy ◽  
Nonlinear Problems ◽  
Optimal Feedback Control ◽  
Policy Space ◽  
Computationally Efficient ◽  
Insertion Point ◽  
Probability Of Success

Abstract We present a computationally efficient approach for the intra-operative update of the feedback control policy for the steerable needle in the presence of the motion uncertainty. The solution to dynamic programming (DP) equations, to obtain the optimal control policy, is difficult or intractable for nonlinear problems such as steering flexible needle in soft tissue. We use the method of approximating Markov chain to approximate the continuous (and controlled) process with its discrete and locally consistent counterpart. This provides the ground to examine the linear programming (LP) approach to solve the imposed DP problem that significantly reduces the computational demand. A concrete example of the two-dimensional (2D) needle steering is considered to investigate the effectiveness of the LP method for both deterministic and stochastic systems. We compare the performance of the LP-based policy with the results obtained through more computationally demanding algorithm, iterative policy space approximation. Finally, the reliability of the LP-based policy dealing with motion and parametric uncertainties as well as the effect of insertion point/angle on the probability of success is investigated.

A Rollout Control Algorithm for Discrete-Time Stochastic Systems

2010 ◽  
Author(s):  
Andreas A. Malikopoulos
Keyword(s):  
Decision Making ◽  
Markov Chain ◽  
Intelligent Systems ◽  
Control Algorithm ◽  
Stochastic Systems ◽  
Intelligent System ◽  
Control Policy ◽  
Suboptimal Control ◽  
Sequential Decision ◽  
Significant Research

The growing demand for making autonomous intelligent systems that can learn how to improve their performance while interacting with their environment has induced significant research on computational cognitive models. Computational intelligence, or rationality, can be achieved by modeling a system and the interaction with its environment through actions, perceptions, and associated costs. A widely adopted paradigm for modeling this interaction is the controlled Markov chain. In this context, the problem is formulated as a sequential decision-making process in which an intelligent system has to select those control actions in several time steps to achieve long-term goals. This paper presents a rollout control algorithm that aims to build an online decision-making mechanism for a controlled Markov chain. The algorithm yields a lookahead suboptimal control policy. Under certain conditions, a theoretical bound on its performance can be established.

On the existence of an optimal feedback control for stochastic systems

2013 ◽  
Vol 49 (11) ◽  
pp. 1456-1464 ◽  
Author(s):  
A. N. Stanzhitskii ◽  
E. A. Samoilenko ◽  
V. V. Mogileva
Keyword(s):  
Feedback Control ◽  
Stochastic Systems ◽  
Optimal Feedback Control ◽  
Optimal Feedback

scholarly journals Causal roles of frontoparietal cortical areas in feedback control of the limb

2020 ◽  
Author(s):  
Tomohiko Takei ◽  
Stephen G. Lomber ◽  
Douglas J. Cook ◽  
Stephen H. Scott
Keyword(s):  
Feedback Control ◽  
Computational Models ◽  
Primary Motor Cortex ◽  
Premotor Cortex ◽  
Control Policy ◽  
Response Speed ◽  
Optimal Feedback Control ◽  
Feedback Gain ◽  
Spatial Accuracy ◽  
Cortical Regions

SummaryGoal-directed motor corrections are surprisingly fast and complex, but little is known on how they are generated by the central nervous system. Here we show that temporary cooling of dorsal premotor cortex (PMd) or parietal area 5 (A5) in behaving monkeys caused impairments in corrective responses to mechanical perturbations of the forelimb. Deactivation of PMd impaired both spatial accuracy and response speed, whereas deactivation of A5 impaired spatial accuracy, but not response speed. Simulations based on optimal feedback control demonstrated that ‘deactivation’ of the control policy (reduction of feedback gain) impaired both spatial accuracy and response speed, whereas ‘deactivation’ in state estimation (reduction of Kalman gain) impaired spatial accuracy but not response speed, paralleling the impairments observed from deactivation of PMd and A5, respectively. Furthermore, combined deactivation of both cortical regions led to additive impairments of individual deactivations, whereas reducing the amount of cooling (i.e. milder cooling) to PMd led to impairments in response speed, but not spatial accuracy, both also predicted by the model simulations. These results provide causal support that higher order motor and somatosensory regions beyond primary somatosensory and primary motor cortex are involved in generating goal-directed motor responses. As well, the computational models suggest that the distinct patterns of impairments associated with these cortical regions reflect their unique functional roles in goal-directed feedback control.

Optimal Feedback Control of a Flexible Needle Under Anatomical Motion Uncertainty

2015 ◽  
Author(s):  
Javad Sovizi ◽  
Suren Kumar ◽  
Venkat Krovi
Keyword(s):  
Dynamic Programming ◽  
Feedback Control ◽  
Programming Problem ◽  
Linear Optimization ◽  
Control Policy ◽  
Optimal Feedback Control ◽  
Patient Specific ◽  
Optimal Feedback ◽  
Planning And Control ◽  
Dynamic Programming Problem

Bevel-tip flexible needles allow for reaching remote/inaccessible organs while avoiding the obstacles (sensitive organs, bones, etc.). Motion planning and control of such systems is a challenging problem due to the uncertainty induced by needle-tissue interactions, anatomical motions (respiratory and cardiac induced motions), imperfect actuation, etc. In this paper, we use an analogy where steering the needle in a soft tissue subject to the uncertain anatomical motions is compared to the Dubins vehicle traveling in the stochastic wind field. Achieving the optimal feedback control policy requires solution of a dynamic programming problem that is often computationally demanding. Efficiency is not central to many optimal control algorithms that often need to be computed only once for a given system/noise statistics. However, intraoperative policy updates may be required for adaptive or patient-specific models. We use the method of approximating Markov chain to approximate the continuous (and controlled) process with its discrete and locally consistent counterpart. We examine the linear programming method of solving the imposed dynamic programming problem that significantly improves the computational efficiency in comparison to the state-of-the-art approaches. In addition, the probability of success and failure are simply the variables of the linear optimization problem and can be directly used for different objective definitions. A numerical example of the 2D needle steering problem is considered to investigate the effectiveness of the proposed method.

Universal Feedback Controllers and Inverse Optimality for Nonlinear Stochastic Systems

2019 ◽  
Vol 142 (2) ◽  
Author(s):  
Wassim M. Haddad ◽  
Xu Jin
Keyword(s):  
Dynamical Systems ◽  
Feedback Control ◽  
Stochastic Control ◽  
Stochastic Systems ◽  
Sufficient Conditions ◽  
Optimal Feedback Control ◽  
Control Law ◽  
Stochastic Dynamical Systems ◽  
Control Lyapunov Function ◽  
Inverse Optimality

Abstract In this paper, we develop a constructive finite time stabilizing feedback control law for stochastic dynamical systems driven by Wiener processes based on the existence of a stochastic control Lyapunov function. In addition, we present necessary and sufficient conditions for continuity of such controllers. Moreover, using stochastic control Lyapunov functions, we construct a universal inverse optimal feedback control law for nonlinear stochastic dynamical systems that possess guaranteed gain and sector margins. An illustrative numerical example involving the control of thermoacoustic instabilities in combustion processes is presented to demonstrate the efficacy of the proposed framework.

An Adaptive Control Policy of Discrete-Time Linear Systems With Random Parameters

1979 ◽  
Vol 101 (4) ◽  
pp. 361-363
Author(s):  
Toshio Yoshimura ◽  
Masanori Kiyota ◽  
Takashi Soeda
Keyword(s):  
Adaptive Control ◽  
Feedback Control ◽  
Linear Systems ◽  
Discrete Time ◽  
Control Policy ◽  
Optimal Feedback Control ◽  
Random Parameters ◽  
Control Cost ◽  
Simulation Results ◽  
Time Linear

An adaptive control policy of discrete-time linear systems with random parameters under a quadratic control cost function is treated. The policy is mainly based on the concept of one-measurement optimal feedback control. Simulation results indicate that the proposed method is superior to the certainty equivalence control.

Sign in / Sign up

Export Citation Format

Share Document