Reinforcement Learning Algorithm for Blood Glucose Control in Diabetic Patients

2015 ◽  
Author(s):  
Mahsa Oroojeni Mohammad Javad ◽  
Stephen Agboola ◽  
Kamal Jethwani ◽  
Ibrahim Zeid ◽  
Sagar Kamarthi
Keyword(s):  
Reinforcement Learning ◽  
Blood Glucose ◽  
Blood Glucose Level ◽  
Infusion Rate ◽  
Insulin Infusion ◽  
Learning Algorithm ◽  
Diabetic Patients ◽  
Basal Rate ◽  
Insulin Infusion Rate ◽  
Reinforcement Learning Algorithm

In this paper a reinforcement learning algorithm is applied to regulating the blood glucose level of Type I diabetic patients using insulin pump. In this approach the agent learns from its exploration and experiences to selects its actions. In the current reinforcement learning algorithm, body weight, A1C level, and physical activity define the state of a diabetic patient. For the agent, insulin dose levels constitute the actions. There are five alternative actions for the agent: (1) raising the insulin infusion rate during 24 hours, (2) keeping it the same, (3) decreasing insulin infusion rate, (4) adjusting basal rate two times during 24 hours, and (5) adjusting basal rate three times during 24 hours. As a result of a patient’s treatment, after each time step t, the reinforcement learning agent receives a numerical reward depending on the response of the patient’s health condition. At each stage the reward is calculated as a function of the deviation of the A1C from its target value. Since reinforcement learning algorithm can select actions that improve patient condition by taking into account delayed effects it has tremendous potential to control blood glucose level in diabetic patients. This research will utilize ten years of clinical data obtained from a hospital.

Reinforcement Learning Based Method for Managing Type 1 Diabetes (Preprint)

2018 ◽  
Author(s):  
Mahsa Oroojeni Mohammad Javad ◽  
Stephen Olusegun Agboola ◽  
Kamal Jethwani ◽  
Ibrahim Zeid ◽  
Sagar Kamarthi
Keyword(s):  
Type 1 Diabetes ◽  
Reinforcement Learning ◽  
Blood Glucose ◽  
Blood Glucose Level ◽  
Diabetic Patient ◽  
Glucose Level ◽  
Dosage Level ◽  
Insulin Dosage ◽  
Diabetic Patients

BACKGROUND Diabetes is a serious chronic disease marked by high levels of blood glucose. It results from issues related to how insulin is produced and/or how insulin functions in the body. In the long run, uncontrolled blood sugar can damage the vessels that supply blood to important organs such as heart, kidneys, eyes, and nerves. Currently there are no effective algorithms to automatically recommend insulin dosage level considering the characteristics of a diabetic patient. OBJECTIVE The objective of this work is to develop and validate a general reinforcement learning framework and a related learning model for personalized treatment and management of Type 1 diabetes and its complications. METHODS This research presents a model-free reinforcement learning (RL) algorithm to recommend insulin level to regulate the blood glucose level of a diabetic patient considering his/her state defined by A1C level, alcohol usage, activity level, and BMI value. In this approach, an RL agent learns from its exploration and response of diabetic patients when they are subject to different actions in terms of insulin dosage level. As a result of a treatment action at time step t, the RL agent receives a numeric reward depending on the response of the patient’s blood glucose level. At each stage the reward for the learning agent is calculated as a function of the difference between the glucose level in the patient body and its target level. The RL algorithm is trained on ten years of the clinical data of 87 patients obtained from the Mass General Hospital. Demographically, 59% of patients are male and 41% of patients are female; the median of age is 54 years and mean is 52.92 years; 86% of patients are white and 47% of 87 patients are married. RESULTS The performance of the algorithm is evaluated on 60 test cases. Further the performance of Support Vector Machine (SVM) has been applied for Lantus class prediction and results has been compared with Q-learning algorithm recommendation. The results show that the RL recommendations of insulin levels for test patients match with the actual prescriptions of the test patients. The RL gave prediction with an accuracy of 88% and SVM shows 80% accuracy. CONCLUSIONS Since the RL algorithm can select actions that improve patient condition by taking into account delayed effects, it has a good potential to control blood glucose level in diabetic patients.

scholarly journals A real-time HIL control system on rotary inverted pendulum hardware platform based on double deep Q-network

2021 ◽  
Vol 54 (3-4) ◽  
pp. 417-428
Author(s):  
Yanyan Dai ◽  
KiDong Lee ◽  
SukGyu Lee
Keyword(s):  
Control System ◽  
Reinforcement Learning ◽  
Inverted Pendulum ◽  
Learning Algorithm ◽  
Deep Understanding ◽  
Control Engineering ◽  
Experience Replay ◽  
Real Hardware ◽  
Rotary Inverted Pendulum ◽  
Reinforcement Learning Algorithm

For real applications, rotary inverted pendulum systems have been known as the basic model in nonlinear control systems. If researchers have no deep understanding of control, it is difficult to control a rotary inverted pendulum platform using classic control engineering models, as shown in section 2.1. Therefore, without classic control theory, this paper controls the platform by training and testing reinforcement learning algorithm. Many recent achievements in reinforcement learning (RL) have become possible, but there is a lack of research to quickly test high-frequency RL algorithms using real hardware environment. In this paper, we propose a real-time Hardware-in-the-loop (HIL) control system to train and test the deep reinforcement learning algorithm from simulation to real hardware implementation. The Double Deep Q-Network (DDQN) with prioritized experience replay reinforcement learning algorithm, without a deep understanding of classical control engineering, is used to implement the agent. For the real experiment, to swing up the rotary inverted pendulum and make the pendulum smoothly move, we define 21 actions to swing up and balance the pendulum. Comparing Deep Q-Network (DQN), the DDQN with prioritized experience replay algorithm removes the overestimate of Q value and decreases the training time. Finally, this paper shows the experiment results with comparisons of classic control theory and different reinforcement learning algorithms.

Sign in / Sign up

Export Citation Format

Share Document