Reinforcement-Learning-Based Virtual Energy Storage System Operation Strategy for Wind Power Forecast Uncertainty Management

Uncertainties related to wind power generation (WPG) restrict its usage. Energy storage systems (ESSs) are key elements employed in managing this uncertainty. This study proposes a reinforcement learning (RL)-based virtual ESS (VESS) operation strategy for WPG forecast uncertainty management. The VESS logically shares a physical ESS to multiple units, while VESS operation reduces the cost barrier of the ESS. In this study, the VESS operation model is suggested considering not only its own operation but also the operation of other units, and the VESS operation problem is formulated as a decision-making problem. To solve this problem, a policy-learning strategy is proposed based on an expected state-action-reward-state-action (SARSA) approach that is robust to variations in uncertainty. Moreover, multi-dimensional clustering is performed according to the WPG forecast data of multiple units to enhance performance. Simulation results using real datasets recorded by the National Renewable Energy Laboratory project of U.S. demonstrate that the proposed strategy provides a near-optimal performance with a less than 2%-point gap with the optimal solution. In addition, the performance of the VESS operation is enhanced by multi-user diversity gain in comparison with individual ESS operation.