Improvement of Air Handling Unit Control Performance Using Reinforcement Learning

2006 ◽  
pp. 168-176
Author(s):  
Sangjo Youk ◽  
Moonseong Kim ◽  
Yangsok Kim ◽  
Gilcheol Park
Keyword(s):  
Reinforcement Learning ◽  
Control Performance ◽  
Air Handling Unit

Reinforcement learning for control: Performance, stability, and deep approximators

2018 ◽  
Vol 46 ◽  
pp. 8-28 ◽  
Author(s):  
Lucian Buşoniu ◽  
Tim de Bruin ◽  
Domagoj Tolić ◽  
Jens Kober ◽  
Ivana Palunko
Keyword(s):  
Reinforcement Learning ◽  
Control Performance

scholarly journals Towards an intelligent HVAC system automation using Reinforcement Learning

2021 ◽  
Vol 2042 (1) ◽  
pp. 012028
Author(s):  
T Schreiber ◽  
A Schwartz ◽  
D Muller
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Good Control ◽  
Problem Formulation ◽  
Pid Controllers ◽  
Computing Power ◽  
Air Handling Unit ◽  
Optimal Control Algorithms ◽  
Control Research ◽  
Building Operation

Abstract HVAC systems are among the biggest energy consumers in buildings and therefore in the focus of optimal control research. In practice, rule-based control and PID controllers are typically used and implemented at the beginning of the building operation. Since this approach neither guarantees optimal or even good control, optimal control algorithms (which can be predictive and adaptive) are in the focus of research. The problem with most of the approaches is that a model of the system is often needed which comes with high engineering efforts. Further, the required computing power can quickly exceed the capacities, even in modern buildings. Therefore, in this paper we investigate the application of a state-of-the-art Reinforcement Learning (RL) algorithm, as a self-calibrating valve controller for two water-air heat exchangers of a real-world air handling unit. We choose a generic problem formulation to pre-train the algorithm with a simulation of an admixing heater and use it to control an injection heater and a throttle cooler. Our results show that after only 70 hours, the control quality significantly increases. Therefore, it seems evident that with pre-trained RL algorithms, a self-improving HVAC automation can be realized with little hardware requirements and without extensive modelling of the system dynamics.

A model free controller based on reinforcement learning for active steering system with uncertainties

2021 ◽  
pp. 095440702199441
Author(s):  
Jintao Zhao ◽  
Shuo Cheng ◽  
Liang Li ◽  
Mingcong Li ◽  
Zhihuang Zhang
Keyword(s):  
Reinforcement Learning ◽  
Steering System ◽  
Effective Control ◽  
Control Performance ◽  
Unknown Parameters ◽  
Network Parameters ◽  
Model Free ◽  
Active Steering ◽  
Model Free Controller ◽  
Active Steering System

Vehicle steering control is crucial to autonomous vehicles. However, unknown parameters and uncertainties of vehicle steering systems bring a great challenge to its control performance, which needs to be tackled urgently. Therefore, this paper proposes a novel model free controller based on reinforcement learning for active steering system with unknown parameters. The model of the active steering system and the Brushless Direct Current (BLDC) motor is built to construct a virtual object in simulations. The agent based on Deep Deterministic Policy Gradient (DDPG) algorithm is built, including actor network and critic network. The rewards from environment are designed to improve the effectiveness of agent. Simulations and testbench experiments are implemented to train the agent and verify the effectiveness of the controller. Results show that the proposed algorithm can acquire the network parameters and achieve effective control performance without any prior knowledges or models. The proposed agent can adapt to different vehicles or active steering systems easily and effectively with only retraining of the network parameters.

scholarly journals Reinforcement Learning based on Deep Deterministic Policy Gradient for Roll Control of Underwater Vehicle

2021 ◽  
Vol 24 (5) ◽  
pp. 558-568
Author(s):  
Su Yong Kim ◽  
Yeon Geol Hwang ◽  
Sung Woong Moon
Keyword(s):  
Reinforcement Learning ◽  
Controller Design ◽  
Control Policy ◽  
Transient State ◽  
Underwater Vehicle ◽  
Control Performance ◽  
Roll Control ◽  
Policy Gradient ◽  
Nonlinear Dynamics Model ◽  
Learning Reinforcement

The existing underwater vehicle controller design is applied by linearizing the nonlinear dynamics model to a specific motion section. Since the linear controller has unstable control performance in a transient state, various studies have been conducted to overcome this problem. Recently, there have been studies to improve the control performance in the transient state by using reinforcement learning. Reinforcement learning can be largely divided into value-based reinforcement learning and policy-based reinforcement learning. In this paper, we propose the roll controller of underwater vehicle based on Deep Deterministic Policy Gradient(DDPG) that learns the control policy and can show stable control performance in various situations and environments. The performance of the proposed DDPG based roll controller was verified through simulation and compared with the existing PID and DQN with Normalized Advantage Functions based roll controllers.

scholarly journals Intelligent Control Strategy for Transient Response of a Variable Geometry Turbocharger System Based on Deep Reinforcement Learning

Processes ◽  
2019 ◽  
Vol 7 (9) ◽  
pp. 601 ◽  
Author(s):  
Hu ◽  
Yang ◽  
Li ◽  
Li ◽  
Bai
Keyword(s):  
Reinforcement Learning ◽  
Control Problems ◽  
Boost Pressure ◽  
Variable Geometry ◽  
Control Performance ◽  
Variable Geometry Turbocharger ◽  
Model Free ◽  
Transient Control ◽  
Gas Exchange System ◽  
Over Time

Deep reinforcement learning (DRL) is an area of machine learning that combines a deep learning approach and reinforcement learning (RL). However, there seem to be few studies that analyze the latest DRL algorithms on real-world powertrain control problems. Meanwhile, the boost control of a variable geometry turbocharger (VGT)-equipped diesel engine is difficult mainly due to its strong coupling with an exhaust gas recirculation (EGR) system and large lag, resulting from time delay and hysteresis between the input and output dynamics of the engine’s gas exchange system. In this context, one of the latest model-free DRL algorithms, the deep deterministic policy gradient (DDPG) algorithm, was built in this paper to develop and finally form a strategy to track the target boost pressure under transient driving cycles. Using a fine-tuned proportion integration differentiation (PID) controller as a benchmark, the results show that the control performance based on the proposed DDPG algorithm can achieve a good transient control performance from scratch by autonomously learning the interaction with the environment, without relying on model supervision or complete environment models. In addition, the proposed strategy is able to adapt to the changing environment and hardware aging over time by adaptively tuning the algorithm in a self-learning manner on-line, making it attractive to real plant control problems whose system consistency may not be strictly guaranteed and whose environment may change over time.

scholarly journals A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR

Energies ◽  
2019 ◽  
Vol 12 (19) ◽  
pp. 3739 ◽  
Author(s):  
Bo Hu ◽  
Jiaxi Li ◽  
Shuang Li ◽  
Jie Yang
Keyword(s):  
Reinforcement Learning ◽  
Diesel Engine ◽  
Control Strategy ◽  
Control Problems ◽  
Variable Geometry ◽  
Control Performance ◽  
Variable Geometry Turbocharger ◽  
End To End ◽  
Boost Control ◽  
Real Plant

Deep reinforcement learning (DRL), which excels at solving a wide variety of Atari and board games, is an area of machine learning that combines the deep learning approach and reinforcement learning (RL). However, to the authors’ best knowledge, there seem to be few studies that apply the latest DRL algorithms on real-world powertrain control problems. If there are any, the requirement of classical model-free DRL algorithms typically for a large number of random exploration in order to realize good control performance makes it almost impossible to implement directly on a real plant. Unlike most of the other DRL studies, whose control strategies can only be trained in a simulation environment—especially when a control strategy has to be learned from scratch—in this study, a hybrid end-to-end control strategy combining one of the latest DRL approaches—i.e., a dueling deep Q-network and traditional Proportion Integration Differentiation (PID) controller—is built, assuming no fidelity simulation model exists. Taking the boost control of a diesel engine with a variable geometry turbocharger (VGT) and cooled (exhaust gas recirculation) EGR as an example, under the common driving cycle, the integral absolute error (IAE) values with the proposed algorithm are improved by 20.66% and 9.7% respectively for the control performance and generality index, compared with a fine-tuned PID benchmark. In addition, the proposed method can also improve system adaptiveness by adding another redundant control module. This makes it attractive to real plant control problems whose simulation models do not exist, and whose environment may change over time.

Sign in / Sign up

Export Citation Format

Share Document