scholarly journals Intelligent Control Strategy for Transient Response of a Variable Geometry Turbocharger System Based on Deep Reinforcement Learning

Processes ◽  
2019 ◽  
Vol 7 (9) ◽  
pp. 601 ◽  
Author(s):  
Hu ◽  
Yang ◽  
Li ◽  
Li ◽  
Bai
Keyword(s):  
Reinforcement Learning ◽  
Control Problems ◽  
Boost Pressure ◽  
Variable Geometry ◽  
Control Performance ◽  
Variable Geometry Turbocharger ◽  
Model Free ◽  
Transient Control ◽  
Gas Exchange System ◽  
Over Time

Deep reinforcement learning (DRL) is an area of machine learning that combines a deep learning approach and reinforcement learning (RL). However, there seem to be few studies that analyze the latest DRL algorithms on real-world powertrain control problems. Meanwhile, the boost control of a variable geometry turbocharger (VGT)-equipped diesel engine is difficult mainly due to its strong coupling with an exhaust gas recirculation (EGR) system and large lag, resulting from time delay and hysteresis between the input and output dynamics of the engine’s gas exchange system. In this context, one of the latest model-free DRL algorithms, the deep deterministic policy gradient (DDPG) algorithm, was built in this paper to develop and finally form a strategy to track the target boost pressure under transient driving cycles. Using a fine-tuned proportion integration differentiation (PID) controller as a benchmark, the results show that the control performance based on the proposed DDPG algorithm can achieve a good transient control performance from scratch by autonomously learning the interaction with the environment, without relying on model supervision or complete environment models. In addition, the proposed strategy is able to adapt to the changing environment and hardware aging over time by adaptively tuning the algorithm in a self-learning manner on-line, making it attractive to real plant control problems whose system consistency may not be strictly guaranteed and whose environment may change over time.

scholarly journals A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR

Energies ◽  
2019 ◽  
Vol 12 (19) ◽  
pp. 3739 ◽  
Author(s):  
Bo Hu ◽  
Jiaxi Li ◽  
Shuang Li ◽  
Jie Yang
Keyword(s):  
Reinforcement Learning ◽  
Diesel Engine ◽  
Control Strategy ◽  
Control Problems ◽  
Variable Geometry ◽  
Control Performance ◽  
Variable Geometry Turbocharger ◽  
End To End ◽  
Boost Control ◽  
Real Plant

Deep reinforcement learning (DRL), which excels at solving a wide variety of Atari and board games, is an area of machine learning that combines the deep learning approach and reinforcement learning (RL). However, to the authors’ best knowledge, there seem to be few studies that apply the latest DRL algorithms on real-world powertrain control problems. If there are any, the requirement of classical model-free DRL algorithms typically for a large number of random exploration in order to realize good control performance makes it almost impossible to implement directly on a real plant. Unlike most of the other DRL studies, whose control strategies can only be trained in a simulation environment—especially when a control strategy has to be learned from scratch—in this study, a hybrid end-to-end control strategy combining one of the latest DRL approaches—i.e., a dueling deep Q-network and traditional Proportion Integration Differentiation (PID) controller—is built, assuming no fidelity simulation model exists. Taking the boost control of a diesel engine with a variable geometry turbocharger (VGT) and cooled (exhaust gas recirculation) EGR as an example, under the common driving cycle, the integral absolute error (IAE) values with the proposed algorithm are improved by 20.66% and 9.7% respectively for the control performance and generality index, compared with a fine-tuned PID benchmark. In addition, the proposed method can also improve system adaptiveness by adding another redundant control module. This makes it attractive to real plant control problems whose simulation models do not exist, and whose environment may change over time.

scholarly journals Neurostimulation Reveals Context-Dependent Arbitration Between Model-Based and Model-Free Reinforcement Learning

2019 ◽  
Vol 29 (11) ◽  
pp. 4850-4862 ◽  
Author(s):  
Sebastian Weissengruber ◽  
Sang Wan Lee ◽  
John P O’Doherty ◽  
Christian C Ruff
Keyword(s):  
Reinforcement Learning ◽  
Neural Mechanism ◽  
Ventrolateral Prefrontal Cortex ◽  
Control Behavior ◽  
Model Based ◽  
Model Free ◽  
Context Dependent ◽  
The Brain ◽  
Over Time ◽  
Environmental Demands

Abstract While it is established that humans use model-based (MB) and model-free (MF) reinforcement learning in a complementary fashion, much less is known about how the brain determines which of these systems should control behavior at any given moment. Here we provide causal evidence for a neural mechanism that acts as a context-dependent arbitrator between both systems. We applied excitatory and inhibitory transcranial direct current stimulation over a region of the left ventrolateral prefrontal cortex previously found to encode the reliability of both learning systems. The opposing neural interventions resulted in a bidirectional shift of control between MB and MF learning. Stimulation also affected the sensitivity of the arbitration mechanism itself, as it changed how often subjects switched between the dominant system over time. Both of these effects depended on varying task contexts that either favored MB or MF control, indicating that this arbitration mechanism is not context-invariant but flexibly incorporates information about current environmental demands.

Optimal Control of Variable Geometry Turbocharged Diesel Engines With Exhaust Gas Recirculation

1999 ◽  
Author(s):  
I. Kolmanovsky ◽  
M. van Nieuwstadt ◽  
P. Moraal
Keyword(s):  
Optimal Control ◽  
Diesel Engines ◽  
Controller Design ◽  
Exhaust Gas Recirculation ◽  
Feedback Controller ◽  
Exhaust Gas ◽  
Variable Geometry ◽  
Variable Geometry Turbocharger ◽  
Transient Control ◽  
Gas Recirculation

Abstract This paper presents results on the optimal transient control of diesel engines with exhaust gas recirculation (EGR) and a variable geometry turbocharger (VGT). The implications of these results for feedback controller design axe discussed.

Simulation of a Diesel Engine with Variable Geometry Turbocharger and Parametric Study of Variable Vane Position on Engine Performance

2017 ◽  
Vol 67 (4) ◽  
pp. 375 ◽  
Author(s):  
Anand Mammen Thomas ◽  
Jensen Samuel J. ◽  
Paul Pramod M. ◽  
A. Ramesh ◽  
R. Murugesan ◽  
...  
Keyword(s):  
Diesel Engine ◽  
Parametric Study ◽  
Thermodynamic Simulation ◽  
Weight Ratio ◽  
Engine Performance ◽  
Boost Pressure ◽  
Variable Geometry ◽  
Variable Geometry Turbocharger ◽  
Additional Degree ◽  
Wide Range

Modelling of a turbocharger is of interest to the engine designer as the work developed by the turbine can be used to drive a compressor coupled to it. This positively influences charge air density and engine power to weight ratio. Variable geometry turbocharger (VGT) additionally has a controllable nozzle ring which is normally electro-pneumatically actuated. This additional degree of freedom offers efficient matching of the effective turbine area for a wide range of engine mass flow rates. Closing of the nozzle ring (vanes tangential to rotor) result in more turbine work and deliver higher boost pressure but it also increases the back pressure on the engine induced by reduced turbine effective area. This adversely affects the net engine torque as the pumping work required increases. Hence, the optimum vane position for a given engine operating point is to be found through simulations or experimentation. A thermodynamic simulation model of a 2.2l 4 cylinder diesel engine was developed for investigation of different control strategies. Model features map based performance prediction of the VGT. Performance of the engine was simulated for steady state operation and validated with experimentation. The results of the parametric study of VGT’s vane position on the engine performance are discussed.

Resonant Intake and Variable Geometry Turbocharging Systems for a V8 Diesel Engine

1983 ◽  
Vol 197 (1) ◽  
pp. 27-36 ◽  
Author(s):  
N Watson
Keyword(s):  
Engine Performance ◽  
Maximum Speed ◽  
Pressure Curve ◽  
Boost Pressure ◽  
Variable Geometry ◽  
Maximum Torque ◽  
Variable Geometry Turbocharger ◽  
Turbine Efficiency ◽  
Mass Flowrate ◽  
Inlet Manifold

A mathematical model is used to analyse the performance of conventional and resonant (Cser type) intake systems on turbocharged six-cylinder and V8 engines. For the V8 the system is slightly less effective than for the six, but is more compact and easily installed within the V of the engine. An optimum system was designed and tested on a V8 engine, with maximum torque b.m.e.p. increasing from 12.0 bar at 1850 rev/min to 12.7 bar at a reduced speed of 1600 rev/min. Engine performance with the resonant system is compared to that with a simple variable-geometry turbocharger turbine, having only one major moving part. A mass-flowrate turndown of 42 per cent was achieved, but with a loss of turbine efficiency largely due to the constraints of adapting an existing turbocharger design. However a flat boost pressure curve was achieved from 1800–2600 rev/min. Maximum torque b.m.e.p. (with conventional inlet manifold) increased to 14 bar at 1600 rev/min. Engine performance at maximum speed (2600 rev/min) was unaffected with either system.

A model free controller based on reinforcement learning for active steering system with uncertainties

2021 ◽  
pp. 095440702199441
Author(s):  
Jintao Zhao ◽  
Shuo Cheng ◽  
Liang Li ◽  
Mingcong Li ◽  
Zhihuang Zhang
Keyword(s):  
Reinforcement Learning ◽  
Steering System ◽  
Effective Control ◽  
Control Performance ◽  
Unknown Parameters ◽  
Network Parameters ◽  
Model Free ◽  
Active Steering ◽  
Model Free Controller ◽  
Active Steering System

Vehicle steering control is crucial to autonomous vehicles. However, unknown parameters and uncertainties of vehicle steering systems bring a great challenge to its control performance, which needs to be tackled urgently. Therefore, this paper proposes a novel model free controller based on reinforcement learning for active steering system with unknown parameters. The model of the active steering system and the Brushless Direct Current (BLDC) motor is built to construct a virtual object in simulations. The agent based on Deep Deterministic Policy Gradient (DDPG) algorithm is built, including actor network and critic network. The rewards from environment are designed to improve the effectiveness of agent. Simulations and testbench experiments are implemented to train the agent and verify the effectiveness of the controller. Results show that the proposed algorithm can acquire the network parameters and achieve effective control performance without any prior knowledges or models. The proposed agent can adapt to different vehicles or active steering systems easily and effectively with only retraining of the network parameters.

A Numerical and Experimental Assessment of the Use of a Turbine Utilizing Splitter Blades for an Automotive Variable Geometry Turbocharger

2014 ◽  
Author(s):  
Jason Walkingshaw ◽  
Stephen Spence ◽  
Dietmar Filsinger ◽  
David Thornhill
Keyword(s):  
Mass Flow ◽  
Numerical Study ◽  
Engine Performance ◽  
Boost Pressure ◽  
Angle Distribution ◽  
Variable Geometry ◽  
Low Velocity ◽  
Turbine Wheel ◽  
Variable Geometry Turbocharger ◽  
Stator Vane

Automotive manufacturers require improved part load engine performance to further improve fuel economy. For a swing vane VGS (Variable Geometry Stator) turbine this means a more closed stator vane, to deal with the low MFRs (Mass Flow Rates), high PRs (Pressure Ratios) and low rotor rotational speeds. During these conditions the turbine is operating at low velocity ratios. As more energy is available at high pressure ratios and during lower turbocharger rotational speeds, a turbine which is efficient at these conditions is desirable. Another key aspect for automotive manufacturers is engine responsiveness. High inertia designs result in “turbo lag” which means an increased time before the target boost pressure is reached. Therefore, designs with improved performance at low velocity ratios, reduced inertia or an increased swallowing capacity are the current targets for turbocharger manufacturers. To try to meet these design targets a CFD (Computational Fluid Dynamics) study was performed on a turbine wheel using splitter blades. A number of parameters were investigated. These included splitter blade merdional length, blade number and blade angle distribution. The numerical study was performed on a scaled automotive VGS. Three different stator vane positions have been analysed. A single passage CFD model was developed and used to provide information on the flow features affecting performance in both the stator vanes and turbine. Following the CFD investigation the design with the best compromise in terms of performance, inertia and increased MFP (Mass Flow Parameter) was selected for manufacture and testing. Tests were performed on a scaled, low temperature turbine test rig. The aerodynamic flow path of the gas stand was the same as that investigated during the CFD. The test results revealed a design which had similar performance at the closed stator vane positions when compared to the baseline wheel. At the maximum MFR stator vane condition a drop of −0.6% pts in efficiency was seen. However, 5.5% increase in MFP was obtained with the additional benefit of a drop in rotor inertia of 3.7%, compared to the baseline wheel.

Model-Based and Model-Free Social Cognition

2019 ◽  
Author(s):  
Leor M Hackel ◽  
Jeffrey Jordan Berg ◽  
Björn Lindström ◽  
David Amodio
Keyword(s):  
Reinforcement Learning ◽  
Social Cognition ◽  
Learning Strategies ◽  
Memory Systems ◽  
Learning Task ◽  
Financial Advisors ◽  
Model Based ◽  
Model Free ◽  
Systems Model ◽  
Task Assessment

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

Sign in / Sign up

Export Citation Format

Share Document