scholarly journals Optimizing the Sensor Placement for Foot Plantar Center of Pressure without Prior Knowledge Using Deep Reinforcement Learning

Sensors ◽  
2020 ◽  
Vol 20 (19) ◽  
pp. 5588
Author(s):  
Cheng-Wu Lin ◽  
Shanq-Jang Ruan ◽  
Wei-Chun Hsu ◽  
Ya-Wen Tu ◽  
Shao-Li Han
Keyword(s):  
Reinforcement Learning ◽  
Prior Knowledge ◽  
Center Of Pressure ◽  
Learning Algorithm ◽  
Sensor Placement ◽  
Ground Truth ◽  
Reward System ◽  
The Self ◽  
Anatomical Area ◽  
Reinforcement Learning Algorithm

We study the foot plantar sensor placement by a deep reinforcement learning algorithm without using any prior knowledge of the foot anatomical area. To apply a reinforcement learning algorithm, we propose a sensor placement environment and reward system that aims to optimize fitting the center of pressure (COP) trajectory during the self-selected speed running task. In this environment, the agent considers placing eight sensors within a 7 × 20 grid coordinate system, and then the final pattern becomes the result of sensor placement. Our results show that this method (1) can generate a sensor placement, which has a low mean square error in fitting ground truth COP trajectory, and (2) robustly discovers the optimal sensor placement in a large number of combinations, which is more than 116 quadrillion. This method is also feasible for solving different tasks, regardless of the self-selected speed running task.

scholarly journals A Reinforcement Learning Algorithm for Automated Detection of Skin Lesions

2021 ◽  
Vol 11 (20) ◽  
pp. 9367
Author(s):  
Usman Ahmad Usmani ◽  
Junzo Watada ◽  
Jafreezal Jaafar ◽  
Izzatdin Abdul Aziz ◽  
Arunava Roy
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Ground Truth ◽  
Skin Lesions ◽  
Action Space ◽  
Gradient Algorithm ◽  
Lesion Segmentation ◽  
Seborrheic Keratosis ◽  
Continuous Action ◽  
Reinforcement Learning Algorithm

Skin cancers are increasing at an alarming rate, and detection in the early stages is essential for advanced treatment. The current segmentation methods have limited labeling ability to the ground truth images due to the numerous noisy expert annotations present in the datasets. The precise boundary segmentation is essential to correctly locate and diagnose the various skin lesions. In this work, the lesion segmentation method is proposed as a Markov decision process. It is solved by training an agent to segment the region using a deep reinforcement-learning algorithm. Our method is similar to the delineation of a region of interest by the physicians. The agent follows a set of serial actions for the region delineation, and the action space is defined as a set of continuous action parameters. The segmentation model learns in continuous action space using the deep deterministic policy gradient algorithm. The proposed method enables continuous improvement in performance as we proceed from coarse segmentation results to finer results. Finally, our proposed model is evaluated on the International Skin Imaging Collaboration (ISIC) 2017 image dataset, Human against Machine (HAM10000), and PH2 dataset. On the ISIC 2017 dataset, the algorithm achieves an accuracy of 96.33% for the naevus cases, 95.39% for the melanoma cases, and 94.27% for the seborrheic keratosis cases. The other metrics are evaluated on these datasets and rank higher when compared with the current state-of-the-art lesion segmentation algorithms.

scholarly journals Hierarchical Reinforcement Learning Based Self-balancing Algorithm for Two-wheeled Robots

2016 ◽  
Vol 10 (1) ◽  
pp. 69-79 ◽  
Author(s):  
Juan Yan ◽  
Huibin Yang
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Weight Vector ◽  
The Self ◽  
Simulation Experiments ◽  
Wheeled Robots ◽  
Reward Function ◽  
Hierarchical Reinforcement Learning ◽  
Balancing Control ◽  
Reinforcement Learning Algorithm

Self-balancing control is the basis for applications of two-wheeled robots. In order to improve the self-balancing of two-wheeled robots, we propose a hierarchical reinforcement learning algorithm for controlling the balance of two-wheeled robots. After describing the subgoals of hierarchical reinforcement learning, we extract features for subgoals, define a feature value vector and its corresponding weight vector, and propose a reward function with additional subgoal reward function. Finally, we give a hierarchical reinforcement learning algorithm for finding the optimal strategy. Simulation experiments show that, the proposed algorithm is more effectiveness than traditional reinforcement learning algorithm in convergent speed. So in our system, the robots can get self-balanced very quickly.

scholarly journals A real-time HIL control system on rotary inverted pendulum hardware platform based on double deep Q-network

2021 ◽  
Vol 54 (3-4) ◽  
pp. 417-428
Author(s):  
Yanyan Dai ◽  
KiDong Lee ◽  
SukGyu Lee
Keyword(s):  
Control System ◽  
Reinforcement Learning ◽  
Inverted Pendulum ◽  
Learning Algorithm ◽  
Deep Understanding ◽  
Control Engineering ◽  
Experience Replay ◽  
Real Hardware ◽  
Rotary Inverted Pendulum ◽  
Reinforcement Learning Algorithm

For real applications, rotary inverted pendulum systems have been known as the basic model in nonlinear control systems. If researchers have no deep understanding of control, it is difficult to control a rotary inverted pendulum platform using classic control engineering models, as shown in section 2.1. Therefore, without classic control theory, this paper controls the platform by training and testing reinforcement learning algorithm. Many recent achievements in reinforcement learning (RL) have become possible, but there is a lack of research to quickly test high-frequency RL algorithms using real hardware environment. In this paper, we propose a real-time Hardware-in-the-loop (HIL) control system to train and test the deep reinforcement learning algorithm from simulation to real hardware implementation. The Double Deep Q-Network (DDQN) with prioritized experience replay reinforcement learning algorithm, without a deep understanding of classical control engineering, is used to implement the agent. For the real experiment, to swing up the rotary inverted pendulum and make the pendulum smoothly move, we define 21 actions to swing up and balance the pendulum. Comparing Deep Q-Network (DQN), the DDQN with prioritized experience replay algorithm removes the overestimate of Q value and decreases the training time. Finally, this paper shows the experiment results with comparisons of classic control theory and different reinforcement learning algorithms.

Sign in / Sign up

Export Citation Format

Share Document