scholarly journals Cooperative secondary voltage control of static converters in a microgrid using model-free reinforcement learning

2019 ◽  
Author(s):  
Edward Smith ◽  
Duane A. Robinson ◽  
Ashish Agalgaonkar
Keyword(s):  
Reinforcement Learning ◽  
Voltage Control ◽  
Model Free ◽  
Static Converters ◽  
Secondary Voltage

scholarly journals Voltage Control-Based Ancillary Service Using Deep Reinforcement Learning

Energies ◽  
2021 ◽  
Vol 14 (8) ◽  
pp. 2274
Author(s):  
Oleh Lukianykhin ◽  
Tetiana Bogodorova
Keyword(s):  
Reinforcement Learning ◽  
Voltage Control ◽  
Batch Size ◽  
Buffer Size ◽  
Time Interval ◽  
State Variables ◽  
Ancillary Service ◽  
Model Free ◽  
Thermostatically Controlled Loads ◽  
Hidden Layer

Ancillary services rely on operating reserves to support an uninterrupted electricity supply that meets demand. One of the hidden reserves of the grid is in thermostatically controlled loads. To efficiently exploit these reserves, a new realization of control of voltage in the allowable range to follow the set power reference is proposed. The proposed approach is based on the deep reinforcement learning (RL) algorithm. Double DQN is utilized because of the proven state-of-the-art level of performance in complex control tasks, native handling of continuous environment state variables, and model-free application of the trained DDQN to the real grid. To evaluate the deep RL control performance, the proposed method was compared with a classic proportional control of the voltage change according to the power reference setup. The solution was validated in setups with a different number of thermostatically controlled loads (TCLs) in a feeder to show its generalization capabilities. In this article, the particularities of deep reinforcement learning application in the power system domain are discussed along with the results achieved by such an RL-powered demand response solution. The tuning of hyperparameters for the RL algorithm was performed to achieve the best performance of the double deep Q-network (DDQN) algorithm. In particular, the influence of a learning rate, a target network update step, network hidden layer size, batch size, and replay buffer size were assessed. The achieved performance is roughly two times better than the competing approach of optimal control selection within the considered time interval of the simulation. The decrease in deviation of the actual power consumption from the reference power profile is demonstrated. The benefit in costs is estimated for the presented voltage control-based ancillary service to show the potential impact.

scholarly journals Management of Voltage Flexibility from Inverter-Based Distributed Generation Using Multi-Agent Reinforcement Learning

Energies ◽  
2021 ◽  
Vol 14 (24) ◽  
pp. 8270
Author(s):  
Nikita Tomin ◽  
Nikolai Voropai ◽  
Victor Kurbatsky ◽  
Christian Rehtanz
Keyword(s):  
Reinforcement Learning ◽  
Voltage Control ◽  
Power Grids ◽  
Droop Control ◽  
Electrical Networks ◽  
Heavy Load ◽  
Market Interaction ◽  
Secondary Voltage ◽  
Multi Agent ◽  
Nodal Voltage

The increase in the use of converter-interfaced generators (CIGs) in today’s electrical grids will require these generators both to supply power and participate in voltage control and provision of grid stability. At the same time, new possibilities of secondary QU droop control in power grids with a large proportion of CIGs (PV panels, wind generators, micro-turbines, fuel cells, and others) open new ways for DSO to increase energy flexibility and maximize hosting capacity. This study extends the existing secondary QU droop control models to enhance the efficiency of CIG integration into electrical networks. The paper presents an approach to decentralized control of secondary voltage through converters based on a multi-agent reinforcement learning (MARL) algorithm. A procedure is also proposed for analyzing hosting capacity and voltage flexibility in a power grid in terms of secondary voltage control. The effectiveness of the proposed static MARL control is demonstrated by the example of a modified IEEE 34-bus test feeder containing CIGs. Experiments have shown that the decentralized approach at issue is effective in stabilizing nodal voltage and preventing overcurrent in lines under various heavy load conditions often caused by active power injections from CIGs themselves and power exchange processes within the TSO/DSO market interaction.

scholarly journals Deep Reinforcement Learning-Based Voltage Control to Deal with Model Uncertainties in Distribution Networks

Energies ◽  
2020 ◽  
Vol 13 (15) ◽  
pp. 3928
Author(s):  
Jean-François Toubeau ◽  
Bashir Bakhshideh Zad ◽  
Martin Hupez ◽  
Zacharie De Grève ◽  
François Vallée
Keyword(s):  
Reinforcement Learning ◽  
Power Flow ◽  
Voltage Control ◽  
Distribution Networks ◽  
Analytical Models ◽  
Physical Parameters ◽  
Voltage Profile ◽  
Promising Alternative ◽  
Model Free ◽  
Linear Behavior

This paper addresses the voltage control problem in medium-voltage distribution networks. The objective is to cost-efficiently maintain the voltage profile within a safe range, in presence of uncertainties in both the future working conditions, as well as the physical parameters of the system. Indeed, the voltage profile depends not only on the fluctuating renewable-based power generation and load demand, but also on the physical parameters of the system components. In reality, the characteristics of loads, lines and transformers are subject to complex and dynamic dependencies, which are difficult to model. In such a context, the quality of the control strategy depends on the accuracy of the power flow representation, which requires to capture the non-linear behavior of the power network. Relying on the detailed analytical models (which are still subject to uncertainties) introduces a high computational power that does not comply with the real-time constraint of the voltage control task. To address this issue, while avoiding arbitrary modeling approximations, we leverage a deep reinforcement learning model to ensure an autonomous grid operational control. Outcomes show that the proposed model-free approach offers a promising alternative to find a compromise between calculation time, conservativeness and economic performance.

Study on Autonomous Decentralized Voltage Control by Reinforcement Learning

2019 ◽  
Vol 139 (2) ◽  
pp. 122-129
Author(s):  
Ryuichiro Takenaka ◽  
Satoshi Takayama ◽  
Atsushi Ishigame
Keyword(s):  
Reinforcement Learning ◽  
Voltage Control

Model-Based and Model-Free Social Cognition

2019 ◽  
Author(s):  
Leor M Hackel ◽  
Jeffrey Jordan Berg ◽  
Björn Lindström ◽  
David Amodio
Keyword(s):  
Reinforcement Learning ◽  
Social Cognition ◽  
Learning Strategies ◽  
Memory Systems ◽  
Learning Task ◽  
Financial Advisors ◽  
Model Based ◽  
Model Free ◽  
Systems Model ◽  
Task Assessment

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

Sign in / Sign up

Export Citation Format

Share Document