scholarly journals Reinforcement Learning in Neurocritical and Neurosurgical Care: Principles and Possible Applications

2021 ◽  
Vol 2021 ◽  
pp. 1-6
Author(s):  
Ying Liu ◽  
Nidan Qiao ◽  
Yuksel Altinel
Keyword(s):  
Reinforcement Learning ◽  
Clinical Care ◽  
Treatment Strategies ◽  
Computational Method ◽  
Dynamic Decision Making ◽  
Medical Studies ◽  
Learning Methods ◽  
Practical Applications ◽  
Observational Dataset ◽  
Optimal Decisions

Dynamic decision-making was essential in the clinical care of surgical patients. Reinforcement learning (RL) algorithm is a computational method to find sequential optimal decisions among multiple suboptimal options. This review is aimed at introducing RL’s basic concepts, including three basic components: the state, the action, and the reward. Most medical studies using reinforcement learning methods were trained on a fixed observational dataset. This paper also reviews the literature of existing practical applications using reinforcement learning methods, which can be further categorized as a statistical RL study and a computational RL study. The review proposes several potential aspects where reinforcement learning can be applied in neurocritical and neurosurgical care. These include sequential treatment strategies of intracranial tumors and traumatic brain injury and intraoperative endoscope motion control. Several limitations of reinforcement learning are representations of basic components, the positivity violation, and validation methods.

Prediction of Disease Comorbidity Using HeteSim Scores based on Multiple Heterogeneous Networks

2019 ◽  
Vol 19 (4) ◽  
pp. 232-241 ◽  
Author(s):  
Xuegong Chen ◽  
Wanwan Shi ◽  
Lei Deng
Keyword(s):  
Protein Interactions ◽  
Experimental Studies ◽  
Treatment Strategies ◽  
Computational Method ◽  
Biological Information ◽  
Support Vector ◽  
Protein Protein Interactions ◽  
Efficient Treatment ◽  
Disease Associations ◽  
Previous State

Background: Accumulating experimental studies have indicated that disease comorbidity causes additional pain to patients and leads to the failure of standard treatments compared to patients who have a single disease. Therefore, accurate prediction of potential comorbidity is essential to design more efficient treatment strategies. However, only a few disease comorbidities have been discovered in the clinic. Objective: In this work, we propose PCHS, an effective computational method for predicting disease comorbidity. Materials and Methods: We utilized the HeteSim measure to calculate the relatedness score for different disease pairs in the global heterogeneous network, which integrates six networks based on biological information, including disease-disease associations, drug-drug interactions, protein-protein interactions and associations among them. We built the prediction model using the Support Vector Machine (SVM) based on the HeteSim scores. Results and Conclusion: The results showed that PCHS performed significantly better than previous state-of-the-art approaches and achieved an AUC score of 0.90 in 10-fold cross-validation. Furthermore, some of our predictions have been verified in literatures, indicating the effectiveness of our method.

scholarly journals Molecular Subtypes and Precision Oncology in Intrahepatic Cholangiocarcinoma

2021 ◽  
Vol 10 (13) ◽  
pp. 2803
Author(s):  
Carolin Czauderna ◽  
Martha M. Kirstein ◽  
Hauke C. Tews ◽  
Arndt Vogel ◽  
Jens U. Marquardt
Keyword(s):  
Clinical Care ◽  
Treatment Options ◽  
Treatment Strategies ◽  
Specific Treatment ◽  
Growth Factor Receptor ◽  
Treatment Modalities ◽  
Precision Oncology ◽  
Recurrence Rates ◽  
Isocitrate Dehydrogenase 1 ◽  
Liver Cancers

Cholangiocarcinomas (CCAs) are the second-most common primary liver cancers. CCAs represent a group of highly heterogeneous tumors classified based on anatomical localization into intra- (iCCA) and extrahepatic CCA (eCCA). In contrast to eCCA, the incidence of iCCA is increasing worldwide. Curative treatment strategies for all CCAs involve oncological resection followed by adjuvant chemotherapy in early stages, whereas chemotherapy is administered at advanced stages of disease. Due to late diagnosis, high recurrence rates, and limited treatment options, the prognosis of patients remains poor. Comprehensive molecular characterization has further revealed considerable heterogeneity and distinct prognostic and therapeutic traits for iCCA and eCCA, indicating that specific treatment modalities are required for different subclasses. Several druggable alterations and oncogenic drivers such as fibroblast growth factor receptor 2 gene fusions and hotspot mutations in isocitrate dehydrogenase 1 and 2 mutations have been identified. Specific inhibitors have demonstrated striking antitumor activity in affected subgroups of patients in phase II and III clinical trials. Thus, improved understanding of the molecular complexity has paved the way for precision oncological approaches. Here, we outline current advances in targeted treatments and immunotherapeutic approaches. In addition, we delineate future perspectives for different molecular subclasses that will improve the clinical care of iCCA patients.

scholarly journals Computational Design of Modular Robots Based on Genetic Algorithm and Reinforcement Learning

Symmetry ◽  
2021 ◽  
Vol 13 (3) ◽  
pp. 471
Author(s):  
Jai Hoon Park ◽  
Kang Hoon Lee
Keyword(s):  
Genetic Algorithm ◽  
Reinforcement Learning ◽  
Design Space ◽  
Learning Algorithm ◽  
Computational Design ◽  
Computational Method ◽  
Learning Ability ◽  
Modular Robots ◽  
Control Mechanisms ◽  
Candidate Structure

Designing novel robots that can cope with a specific task is a challenging problem because of the enormous design space that involves both morphological structures and control mechanisms. To this end, we present a computational method for automating the design of modular robots. Our method employs a genetic algorithm to evolve robotic structures as an outer optimization, and it applies a reinforcement learning algorithm to each candidate structure to train its behavior and evaluate its potential learning ability as an inner optimization. The size of the design space is reduced significantly by evolving only the robotic structure and by performing behavioral optimization using a separate training algorithm compared to that when both the structure and behavior are evolved simultaneously. Mutual dependence between evolution and learning is achieved by regarding the mean cumulative rewards of a candidate structure in the reinforcement learning as its fitness in the genetic algorithm. Therefore, our method searches for prospective robotic structures that can potentially lead to near-optimal behaviors if trained sufficiently. We demonstrate the usefulness of our method through several effective design results that were automatically generated in the process of experimenting with actual modular robotics kit.

scholarly journals Modulation of optical absorption in m-Fe1−xRuxS2 and exploring stability in new m-RuS2

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
H. Joshi ◽  
M. Ram ◽  
N. Limbu ◽  
D. P. Rai ◽  
B. Thapa ◽  
...  
Keyword(s):  
Optical Absorption ◽  
Phonon Dispersion ◽  
Applied Pressure ◽  
Orthorhombic Phase ◽  
Zero Point ◽  
Computational Method ◽  
Crystal Field Theory ◽  
Point Energy ◽  
Practical Applications ◽  
New Phase

AbstractA first-principle computational method has been used to investigate the effects of Ru dopants on the electronic and optical absorption properties of marcasite FeS2. In addition, we have also revealed a new marcasite phase in RuS2, unlike most studied pyrite structures. The new phase has fulfilled all the necessary criteria of structural stability and its practical existence. The transition pressure of 8 GPa drives the structural change from pyrite to orthorhombic phase in RuS2. From the thermodynamical calculation, we have reported the stability of new-phase under various ranges of applied pressure and temperature. Further, from the results of phonon dispersion calculated at Zero Point Energy, pyrite structure exhibits ground state stability and the marcasite phase has all modes of frequencies positive. The newly proposed phase is a semiconductor with a band gap comparable to its pyrite counterpart but vary in optical absorption by around 106 cm−1. The various Ru doped structures have also shown similar optical absorption spectra in the same order of magnitude. We have used crystal field theory to explain high optical absorption which is due to the involvement of different electronic states in formation of electronic and optical band gaps. Lӧwdin charge analysis is used over the customarily Mulliken charges to predict 89% of covalence in the compound. Our results indicate the importance of new phase to enhance the efficiency of photovoltaic materials for practical applications.

Hierarchical Reinforcement Learning

2021 ◽  
Vol 54 (5) ◽  
pp. 1-35
Author(s):  
Shubham Pateria ◽  
Budhitama Subagdja ◽  
Ah-hwee Tan ◽  
Chai Quek
Keyword(s):  
Reinforcement Learning ◽  
Future Research ◽  
Comprehensive Overview ◽  
Open Problems ◽  
Practical Applications ◽  
Hierarchical Reinforcement Learning ◽  
The Past ◽  
Agent Learning ◽  
Multi Agent ◽  
Supplementary Material

Hierarchical Reinforcement Learning (HRL) enables autonomous decomposition of challenging long-horizon decision-making tasks into simpler subtasks. During the past years, the landscape of HRL research has grown profoundly, resulting in copious approaches. A comprehensive overview of this vast landscape is necessary to study HRL in an organized manner. We provide a survey of the diverse HRL approaches concerning the challenges of learning hierarchical policies, subtask discovery, transfer learning, and multi-agent learning using HRL. The survey is presented according to a novel taxonomy of the approaches. Based on the survey, a set of important open problems is proposed to motivate the future research in HRL. Furthermore, we outline a few suitable task domains for evaluating the HRL approaches and a few interesting examples of the practical applications of HRL in the Supplementary Material.

Adaptive stock trading strategies with deep reinforcement learning methods

2020 ◽  
Vol 538 ◽  
pp. 142-158 ◽  
Author(s):  
Xing Wu ◽  
Haolei Chen ◽  
Jianjia Wang ◽  
Luigi Troiano ◽  
Vincenzo Loia ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Trading Strategies ◽  
Stock Trading ◽  
Learning Methods

A Model-Based Factored Bayesian Reinforcement Learning Approach

2014 ◽  
Vol 513-517 ◽  
pp. 1092-1095
Author(s):  
Bo Wu ◽  
Yan Peng Feng ◽  
Hong Yan Zheng
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Iteration Algorithm ◽  
Value Iteration ◽  
Practical Applications ◽  
Model Based ◽  
Online Planning ◽  
Bayesian Reinforcement Learning ◽  
Bayesian Inference Method ◽  
Unknown Structure

Bayesian reinforcement learning has turned out to be an effective solution to the optimal tradeoff between exploration and exploitation. However, in practical applications, the learning parameters with exponential growth are the main impediment for online planning and learning. To overcome this problem, we bring factored representations, model-based learning, and Bayesian reinforcement learning together in a new approach. Firstly, we exploit a factored representation to describe the states to reduce the size of learning parameters, and adopt Bayesian inference method to learn the unknown structure and parameters simultaneously. Then, we use an online point-based value iteration algorithm to plan and learn. The experimental results show that the proposed approach is an effective way for improving the learning efficiency in large-scale state spaces.

Sign in / Sign up

Export Citation Format

Share Document