scholarly journals Controlling Blood Glucose For Patients With Type 1 DiabetesUsing Deep Reinforcement Learning – The Influence OfChanging The Reward Function

2020 ◽  
Vol 1 ◽  
pp. 6
Author(s):  
Miguel Angel Tejedor Hernandez ◽  
Jonas Nordhaug Myhre
Keyword(s):  
Type 1 Diabetes ◽  
Reinforcement Learning ◽  
Blood Glucose ◽  
In Silico ◽  
Critical Component ◽  
Promising Direction ◽  
Reward Function ◽  
Reward Functions

Reinforcement learning (RL) is a promising direction in adaptive and personalized type 1 diabetes (T1D) treatment. However, the reward function – a most critical component in RL – is a component that is in most cases hand designed and often overlooked. In this paper we show that different reward functions can dramatically influence the final result when using RL to treat in-silico T1D patients.

scholarly journals Control of Blood Glucose Level for Type 1 Diabetes Mellitus using Improved Hovorka Equations: Comparison between Clinical and In-Silico Works

2020 ◽  
Author(s):  
Nur’Amanina Mohd Sohadi ◽  
Ayub Md Som ◽  
Noor Shafina Mohd Nor ◽  
Nur Farhana Mohd Yusof ◽  
Sherif Abdulbari Ali ◽  
...  
Keyword(s):  
Diabetes Mellitus ◽  
Type 1 Diabetes ◽  
Blood Glucose ◽  
Blood Glucose Level ◽  
Type 1 Diabetes Mellitus ◽  
Glucose Level ◽  
In Silico ◽  
Artificial Pancreas ◽  
The Body

AbstractBackgroundType 1 diabetes mellitus (T1DM) occurs due to inability of the body to produce sufficient amount of insulin to regulate blood glucose level (BGL) at normoglycemic range between 4.0 to 7.0 mmol/L. Thus, T1DM patients require to do self-monitoring blood glucose (SMBG) via finger pricks and depend on exogenous insulin injection to maintain their BGL which is very painful and exasperating. Ongoing works on artificial pancreas device nowadays focus primarily on a computer algorithm which is programmed into the controller device. This study aims to simulate so-called improved equations from the Hovorka model using actual patients’ data through in-silico works and compare its findings with the clinical works.MethodsThe study mainly focuses on computer simulation in MATLAB using improved Hovorka equations in order to control the BGL in T1DM. The improved equations can be found in three subsystems namely; glucose, insulin and insulin action subsystems. CHO intakes were varied during breakfast, lunch and dinner times for three consecutive days. Simulated data are compared with the actual patients’ data from the clinical works.ResultsResult revealed that when the patient took 36.0g CHO during breakfast and lunch, the insulin administered was 0.1U/min in order to maintain the blood glucose level (BGL) in the safe range after meal; while during dinner time, 0.083U/min to 0.1 U/min of insulins were administered in order to regulate 45.0g CHO taken during meal. The basal insulin was also injected at 0.066U/min upon waking up time in the early morning. The BGL was able to remain at normal range after each meal during in-silico works compared to clinical works.ConclusionsThis study proved that the improved Hovorka equations via in-silico works can be employed to model the effect of meal disruptions on T1DM patients, as it demonstrated better control as compared to the clinical works.

Reinforcement Learning Based Method for Managing Type 1 Diabetes (Preprint)

2018 ◽  
Author(s):  
Mahsa Oroojeni Mohammad Javad ◽  
Stephen Olusegun Agboola ◽  
Kamal Jethwani ◽  
Ibrahim Zeid ◽  
Sagar Kamarthi
Keyword(s):  
Type 1 Diabetes ◽  
Reinforcement Learning ◽  
Blood Glucose ◽  
Blood Glucose Level ◽  
Diabetic Patient ◽  
Glucose Level ◽  
Dosage Level ◽  
Insulin Dosage ◽  
Diabetic Patients

BACKGROUND Diabetes is a serious chronic disease marked by high levels of blood glucose. It results from issues related to how insulin is produced and/or how insulin functions in the body. In the long run, uncontrolled blood sugar can damage the vessels that supply blood to important organs such as heart, kidneys, eyes, and nerves. Currently there are no effective algorithms to automatically recommend insulin dosage level considering the characteristics of a diabetic patient. OBJECTIVE The objective of this work is to develop and validate a general reinforcement learning framework and a related learning model for personalized treatment and management of Type 1 diabetes and its complications. METHODS This research presents a model-free reinforcement learning (RL) algorithm to recommend insulin level to regulate the blood glucose level of a diabetic patient considering his/her state defined by A1C level, alcohol usage, activity level, and BMI value. In this approach, an RL agent learns from its exploration and response of diabetic patients when they are subject to different actions in terms of insulin dosage level. As a result of a treatment action at time step t, the RL agent receives a numeric reward depending on the response of the patient’s blood glucose level. At each stage the reward for the learning agent is calculated as a function of the difference between the glucose level in the patient body and its target level. The RL algorithm is trained on ten years of the clinical data of 87 patients obtained from the Mass General Hospital. Demographically, 59% of patients are male and 41% of patients are female; the median of age is 54 years and mean is 52.92 years; 86% of patients are white and 47% of 87 patients are married. RESULTS The performance of the algorithm is evaluated on 60 test cases. Further the performance of Support Vector Machine (SVM) has been applied for Lantus class prediction and results has been compared with Q-learning algorithm recommendation. The results show that the RL recommendations of insulin levels for test patients match with the actual prescriptions of the test patients. The RL gave prediction with an accuracy of 88% and SVM shows 80% accuracy. CONCLUSIONS Since the RL algorithm can select actions that improve patient condition by taking into account delayed effects, it has a good potential to control blood glucose level in diabetic patients.

scholarly journals The Financial Impact of Inaccurate Blood Glucose Monitoring Systems

2017 ◽  
Vol 12 (2) ◽  
pp. 318-324 ◽  
Author(s):  
Kurt Fortwaengler ◽  
Enrique Campos-Náñez ◽  
Christopher G. Parkin ◽  
Marc D. Breton
Keyword(s):  
Type 1 Diabetes ◽  
Blood Glucose ◽  
In Silico ◽  
Glucose Monitoring ◽  
Financial Impact ◽  
Blood Glucose Monitoring ◽  
Entire Group ◽  
In Silico Study ◽  
Average System

Objective: An in silico study of type 1 diabetes (T1DM) patients utilized the UVA-PADOVA Type 1 Diabetes Simulator to assess the effect of patient blood glucose monitoring (BGM) system accuracy on clinical outcomes. We applied these findings to assess the financial impact of BGM system inaccuracy. Methods: The study included 43 BGM systems previously assessed for accuracy according to ISO 15197:2003 and ISO 15197:2013 criteria. Glycemic responses for the 100 in silico adult T1DM subjects were generated, using each meter. Changes in estimated HbA1c, severe hypoglycemic events, and health care resource utilization were computed for each simulation. The HbA1c Translator modeling approach was used to calculate the financial impact of these changes. Results: The average cost of inaccuracy associated with the entire group of BGM systems was £155 per patient year (PPY). The average additional cost of BGM systems not meeting the ISO 15197:2003 standard was an estimated £178 PPY more than an average system that fulfills the standard and an estimated £235 PPY more than an average system that appears to meet the ISO 15197:2013 standard. Conclusion: There is a clear relationship between BGM system accuracy and cost, with the highest costs being associated with BGM systems not meeting the ISO 15197:2003 standard. Lower costs are associated with systems meeting the ISO 15197:2013 system accuracy criteria. Using BGM systems that meet the system accuracy criteria of the ISO 15197:2013 standard can help reduce the clinical and financial consequences associated with inaccuracy of BGM devices.

scholarly journals A Reinforcement Learning–Based Method for Management of Type 1 Diabetes: Exploratory Study

JMIR Diabetes ◽  
10.2196/12905 ◽  
2019 ◽  
Vol 4 (3) ◽  
pp. e12905 ◽  
Author(s):  
Mahsa Oroojeni Mohammad Javad ◽  
Stephen Olusegun Agboola ◽  
Kamal Jethwani ◽  
Abe Zeid ◽  
Sagar Kamarthi
Keyword(s):  
Type 1 Diabetes ◽  
Reinforcement Learning ◽  
Blood Glucose ◽  
Blood Glucose Level ◽  
Glucose Level ◽  
Clinical Data ◽  
Exploratory Study ◽  
Insulin Dosage

Background Type 1 diabetes mellitus (T1DM) is characterized by chronic insulin deficiency and consequent hyperglycemia. Patients with T1DM require long-term exogenous insulin therapy to regulate blood glucose levels and prevent the long-term complications of the disease. Currently, there are no effective algorithms that consider the unique characteristics of T1DM patients to automatically recommend personalized insulin dosage levels. Objective The objective of this study was to develop and validate a general reinforcement learning (RL) framework for the personalized treatment of T1DM using clinical data. Methods This research presents a model-free data-driven RL algorithm, namely Q-learning, that recommends insulin doses to regulate the blood glucose level of a T1DM patient, considering his or her state defined by glycated hemoglobin (HbA1c) levels, body mass index, engagement in physical activity, and alcohol usage. In this approach, the RL agent identifies the different states of the patient by exploring the patient’s responses when he or she is subjected to varying insulin doses. On the basis of the result of a treatment action at time step t, the RL agent receives a numeric reward, positive or negative. The reward is calculated as a function of the difference between the actual blood glucose level achieved in response to the insulin dose and the targeted HbA1c level. The RL agent was trained on 10 years of clinical data of patients treated at the Mass General Hospital. Results A total of 87 patients were included in the training set. The mean age of these patients was 53 years, 59% (51/87) were male, 86% (75/87) were white, and 47% (41/87) were married. The performance of the RL agent was evaluated on 60 test cases. RL agent–recommended insulin dosage interval includes the actual dose prescribed by the physician in 53 out of 60 cases (53/60, 88%). Conclusions This exploratory study demonstrates that an RL algorithm can be used to recommend personalized insulin doses to achieve adequate glycemic control in patients with T1DM. However, further investigation in a larger sample of patients is needed to confirm these findings.

scholarly journals In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus

2020 ◽  
Vol 10 (18) ◽  
pp. 6350
Author(s):  
Jonas Nordhaug Myhre ◽  
Miguel Tejedor ◽  
Ilkka Kalervo Launonen ◽  
Anas El Fathi ◽  
Fred Godtliebsen
Keyword(s):  
Diabetes Mellitus ◽  
Type 1 Diabetes ◽  
Reinforcement Learning ◽  
Blood Glucose ◽  
State Of The Art ◽  
Blood Glucose Control ◽  
Glucose Regulation ◽  
Gradient Algorithms ◽  
Policy Gradient

In this paper, we test and evaluate policy gradient reinforcement learning for automated blood glucose control in patients with Type 1 Diabetes Mellitus. Recent research has shown that reinforcement learning is a promising approach to accommodate the need for individualized blood glucose level control algorithms. The motivation for using policy gradient algorithms comes from the fact that adaptively administering insulin is an inherently continuous task. Policy gradient algorithms are known to be superior in continuous high-dimensional control tasks. Previously, most of the approaches for automated blood glucose control using reinforcement learning has used a finite set of actions. We use the Trust-Region Policy Optimization algorithm in this work. It represents the state of the art for deep policy gradient algorithms. The experiments are carried out in-silico using the Hovorka model, and stochastic behavior is modeled through simulated carbohydrate counting errors to illustrate the full potential of the framework. Furthermore, we use a model-free approach where no prior information about the patient is given to the algorithm. Our experiments show that the reinforcement learning agent is able to compete with and sometimes outperform state-of-the-art model predictive control in blood glucose regulation.

scholarly journals Control of Blood Glucose for Type-1 Diabetes by Using Reinforcement Learning with Feedforward Algorithm

2018 ◽  
Vol 2018 ◽  
pp. 1-8 ◽  
Author(s):  
Phuong D. Ngo ◽  
Susan Wei ◽  
Anna Holubová ◽  
Jan Muzik ◽  
Fred Godtliebsen
Keyword(s):  
Type 1 Diabetes ◽  
Control System ◽  
Reinforcement Learning ◽  
Blood Glucose ◽  
State Variables ◽  
Meal Time ◽  
Feedforward Controller ◽  
Bolus Insulin ◽  
Nonlinear Nature

Background. Type-1 diabetes is a condition caused by the lack of insulin hormone, which leads to an excessive increase in blood glucose level. The glucose kinetics process is difficult to control due to its complex and nonlinear nature and with state variables that are difficult to measure. Methods. This paper proposes a method for automatically calculating the basal and bolus insulin doses for patients with type-1 diabetes using reinforcement learning with feedforward controller. The algorithm is designed to keep the blood glucose stable and directly compensate for the external events such as food intake. Its performance was assessed using simulation on a blood glucose model. The usage of the Kalman filter with the controller was demonstrated to estimate unmeasurable state variables. Results. Comparison simulations between the proposed controller with the optimal reinforcement learning and the proportional-integral-derivative controller show that the proposed methodology has the best performance in regulating the fluctuation of the blood glucose. The proposed controller also improved the blood glucose responses and prevented hypoglycemia condition. Simulation of the control system in different uncertain conditions provided insights on how the inaccuracies of carbohydrate counting and meal-time reporting affect the performance of the control system. Conclusion. The proposed controller is an effective tool for reducing postmeal blood glucose rise and for countering the effects of external known events such as meal intake and maintaining blood glucose at a healthy level under uncertainties.

scholarly journals In Silico Modeling of Minimal Effective Insulin Doses Using the UVA/PADOVA Type 1 Diabetes Simulator

2017 ◽  
Vol 12 (2) ◽  
pp. 376-380 ◽  
Author(s):  
Enrique Campos-Náñez ◽  
Jennifer E. Layne ◽  
Howard C. Zisser
Keyword(s):  
Type 1 Diabetes ◽  
Blood Glucose ◽  
In Silico ◽  
Basal Insulin ◽  
Insulin Infusion ◽  
Glucose Levels ◽  
In Silico Modeling ◽  
Adolescents And Adults ◽  
Bolus Insulin

Background: The objective of this study was to identify the minimum basal insulin infusion rates and bolus insulin doses that would result in clinically relevant changes in blood glucose levels in the most insulin sensitive subjects with type 1 diabetes. Methods: The UVA/PADOVA Type 1 Diabetes Simulator in silico population of children, adolescents, and adults was administered a basal insulin infusion rate to maintain blood glucose concentrations at 120 mg/dL (6.7 mmol/L). Two scenarios were modeled independently after 1 hour of simulated time: (1) basal insulin infusion rates in increments of 0.01 U/h were administered and (2) bolus doses in increments of 0.01 U were injected. Subjects were observed for 4 hours to determine insulin delivery required to change blood glucose by 12.5 mg/dL (0.7 mmol/L) and 25 mg/dL (1.4 mmol/L) in only 5% of the in silico population. Results: The basal insulin infusion rates required to change blood glucose by 12.5 mg/dL and 25 mg/dL in 5% of children, adolescents, and adults were 0.03, 0.11, and 0.10 U/h and 0.06, 0.21, and 0.19 U/h, respectively. The bolus insulin doses required to change blood glucose by the target amounts in the respective populations were 0.10, 0.28, and 0.30 U and 0.19, 0.55, and 0.60 U. Conclusions: In silico modeling suggests that only a very small percentage of individuals with type 1 diabetes, corresponding to children with high insulin sensitivity and low body weight, will exhibit a clinically relevant change in blood glucose with very low basal insulin rate changes or bolus doses.

Sign in / Sign up

Export Citation Format

Share Document