scholarly journals Multi-Agent Reinforcement Learning Approach for Residential Microgrid Energy Scheduling

Energies ◽  
2019 ◽  
Vol 13 (1) ◽  
pp. 123 ◽  
Author(s):  
Xiaohan Fang ◽  
Jinkuan Wang ◽  
Guanru Song ◽  
Yinghua Han ◽  
Qiang Zhao ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Single Agent ◽  
New Paradigm ◽  
Agent Based ◽  
Vehicle To Grid ◽  
Market Participants ◽  
Home Energy Management ◽  
Market Operation ◽  
Multi Agent ◽  
Home Energy Management System

Residential microgrid is widely considered as a new paradigm of the home energy management system. The complexity of Microgrid Energy Scheduling (MES) is increasing with the integration of Electric Vehicles (EVs) and Renewable Generations (RGs). Moreover, it is challenging to determine optimal scheduling strategies to guarantee the efficiency of the microgrid market and to balance all market participants’ benefits. In this paper, a Multi-Agent Reinforcement Learning (MARL) approach for residential MES is proposed to promote the autonomy and fairness of microgrid market operation. First, a multi-agent based residential microgrid model including Vehicle-to-Grid (V2G) and RGs is constructed and an auction-based microgrid market is built. Then, distinguish from Single-Agent Reinforcement Learning (SARL), MARL can achieve distributed autonomous learning for each agent and realize the equilibrium of all agents’ benefits, therefore, we formulate an equilibrium-based MARL framework according to each participant’ market orientation. Finally, to guarantee the fairness and privacy of the MARL process, we proposed an improved optimal Equilibrium Selection-MARL (ES-MARL) algorithm based on two mechanisms, private negotiation and maximum average reward. Simulation results demonstrate the overall performance and efficiency of proposed MARL are superior to that of SARL. Besides, it is verified that the improved ES-MARL can get higher average profit to balance all agents.

scholarly journals Multi-Agent Reinforcement Learning: A Review of Challenges and Applications

2021 ◽  
Vol 11 (11) ◽  
pp. 4948
Author(s):  
Lorenzo Canese ◽  
Gian Carlo Cardarilli ◽  
Luca Di Di Nunzio ◽  
Rocco Fazzolari ◽  
Daniele Giardino ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Mathematical Models ◽  
Learning Algorithms ◽  
Single Agent ◽  
Critical Issues ◽  
Multi Agent ◽  
Pros And Cons ◽  
Application Fields

In this review, we present an analysis of the most used multi-agent reinforcement learning algorithms. Starting with the single-agent reinforcement learning algorithms, we focus on the most critical issues that must be taken into account in their extension to multi-agent scenarios. The analyzed algorithms were grouped according to their features. We present a detailed taxonomy of the main multi-agent approaches proposed in the literature, focusing on their related mathematical models. For each algorithm, we describe the possible application fields, while pointing out its pros and cons. The described multi-agent algorithms are compared in terms of the most important characteristics for multi-agent reinforcement learning applications—namely, nonstationarity, scalability, and observability. We also describe the most common benchmark environments used to evaluate the performances of the considered methods.

scholarly journals On-Demand Channel Bonding in Heterogeneous WLANs: A Multi-Agent Deep Reinforcement Learning Approach

Sensors ◽  
2020 ◽  
Vol 20 (10) ◽  
pp. 2789 ◽  
Author(s):  
Hang Qi ◽  
Hao Huang ◽  
Zhiqun Hu ◽  
Xiangming Wen ◽  
Zhaoming Lu
Keyword(s):  
Reinforcement Learning ◽  
Transmission Rate ◽  
Single Agent ◽  
Time Of Day ◽  
Action Space ◽  
Traffic Load ◽  
Traffic Demand ◽  
Channel Bonding ◽  
On Demand ◽  
Multi Agent

In order to meet the ever-increasing traffic demand of Wireless Local Area Networks (WLANs), channel bonding is introduced in IEEE 802.11 standards. Although channel bonding effectively increases the transmission rate, the wider channel reduces the number of non-overlapping channels and is more susceptible to interference. Meanwhile, the traffic load differs from one access point (AP) to another and changes significantly depending on the time of day. Therefore, the primary channel and channel bonding bandwidth should be carefully selected to meet traffic demand and guarantee the performance gain. In this paper, we proposed an On-Demand Channel Bonding (O-DCB) algorithm based on Deep Reinforcement Learning (DRL) for heterogeneous WLANs to reduce transmission delay, where the APs have different channel bonding capabilities. In this problem, the state space is continuous and the action space is discrete. However, the size of action space increases exponentially with the number of APs by using single-agent DRL, which severely affects the learning rate. To accelerate learning, Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is used to train O-DCB. Real traffic traces collected from a campus WLAN are used to train and test O-DCB. Simulation results reveal that the proposed algorithm has good convergence and lower delay than other algorithms.

A multi-agent based integrated volt-var optimization engine for fast vehicle-to-grid reactive power dispatch and electric vehicle coordination

2018 ◽  
Vol 229 ◽  
pp. 96-110 ◽  
Author(s):  
Wenjie Zhang ◽  
Oktoviano Gandhi ◽  
Hao Quan ◽  
Carlos D. Rodríguez-Gallegos ◽  
Dipti Srinivasan
Keyword(s):  
Electric Vehicle ◽  
Reactive Power ◽  
Agent Based ◽  
Vehicle To Grid ◽  
Power Dispatch ◽  
Multi Agent ◽  
Reactive Power Dispatch

scholarly journals Implementation of home energy management system based on reinforcement learning

2022 ◽  
Vol 8 ◽  
pp. 560-566
Author(s):  
Ejaz Ul Haq ◽  
Cheng Lyu ◽  
Peng Xie ◽  
Shuo Yan ◽  
Fiaz Ahmad ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Energy Management ◽  
Management System ◽  
Energy Management System ◽  
Home Energy Management ◽  
Home Energy Management System

Multi-agent reinforcement learning using ordinal action selection and approximate policy iteration

2016 ◽  
Vol 14 (06) ◽  
pp. 1650053
Author(s):  
Daxue Liu ◽  
Jun Wu ◽  
Xin Xu
Keyword(s):  
Reinforcement Learning ◽  
Single Agent ◽  
Action Selection ◽  
Policy Iteration ◽  
Common Interest ◽  
Policy Space ◽  
Markov Games ◽  
Approximate Policy Iteration ◽  
Multi Agent ◽  
Agent Coordination

Multi-agent reinforcement learning (MARL) provides a useful and flexible framework for multi-agent coordination in uncertain dynamic environments. However, the generalization ability and scalability of algorithms to large problem sizes, already problematic in single-agent RL, is an even more formidable obstacle in MARL applications. In this paper, a new MARL method based on ordinal action selection and approximate policy iteration called OAPI (Ordinal Approximate Policy Iteration), is presented to address the scalability issue of MARL algorithms in common-interest Markov Games. In OAPI, an ordinal action selection and learning strategy is integrated with distributed approximate policy iteration not only to simplify the policy space and eliminate the conflicts in multi-agent coordination, but also to realize the approximation of near-optimal policies for Markov Games with large state spaces. Based on the simplified policy space using ordinal action selection, the OAPI algorithm implements distributed approximate policy iteration utilizing online least-squares policy iteration (LSPI). This resulted in multi-agent coordination with good convergence properties with reduced computational complexity. The simulation results of a coordinated multi-robot navigation task illustrate the feasibility and effectiveness of the proposed approach.

BDI Approach to Build a Single Agent of a Distributed Multi-Agent System

2018 ◽  
pp. 24-49
Author(s):  
Maryam Ebrahimi
Keyword(s):  
Swot Analysis ◽  
Planning Process ◽  
Single Agent ◽  
Defensive Strategy ◽  
Diversification Strategy ◽  
Agent Based ◽  
Autonomous Action ◽  
System Validation ◽  
Multi Agent ◽  
Technology Strategies

The main purpose of this study is to describe and analyze an agent from a distributed multi-agent based system (ABS) according to the BDI architecture. This agent is capable of autonomous action to propose general technology strategies (TSs) in renewable energy SMEs based on a set of rules and interacts with a core agent in multi ABS. The recognition of internal strengths and weaknesses as well as external opportunities and threats takes place on the basis of technological SWOT-analysis. Proposed TSs are categorized into four types: aggressive strategy, turnaround oriented strategy, diversification strategy, and defensive strategy. Agent architecture in terms of three abstraction layers called psychological, theoretical, and implementation is explained. And after system validation by experts, some program codes and output results of this agent are presented. This system provides information to facilitate the TS planning process to be carried out effectively.

An Effectiveness Modelling Approach for IoT-Based Smart Grids

2021 ◽  
pp. 92-111
Author(s):  
Dongming Fan ◽  
Yi Ren ◽  
Qiang Feng
Keyword(s):  
Smart Grid ◽  
Smart Grids ◽  
Information Technologies ◽  
Self Healing ◽  
New Paradigm ◽  
Agent Based ◽  
Efficient Energy ◽  
Real Time Analysis ◽  
Multi Agent

The smart grid is a new paradigm that enables highly efficient energy production, transport, and consumption along the whole chain from the source to the user. The smart grid is the combination of classical power grid with emerging communication and information technologies. IoT-based smart grid will be one of the largest instantiations of the IoT in the future. The effectiveness of IoT-based smart grid is mainly reflected in observability, real-time analysis, decision-making, and self-healing. A proper effectiveness modeling approach should maintain the reliability and maintainability of IoT-based smart grids. In this chapter, a multi-agent-based approach is proposed to model the architecture of IoT-based smart grids. Based on the agent framework, certain common types of agents are provided to describe the operation and restoration process of smart grids. A case study is demonstrated to model an IoT-based smart grid with restoration, and the interactive process with agents is proposed simultaneously.

Intelligent Agents in a Trust Environment

2002 ◽  
pp. 98-108
Author(s):  
Rahul Singh ◽  
Mark A. Gill
Keyword(s):  
Distributed System ◽  
Intelligent Agents ◽  
Single Agent ◽  
Intelligent Agent ◽  
The Internet ◽  
Commercial Activity ◽  
Agent Based ◽  
Trust Relationships ◽  
Multi Agent ◽  
Oriented Activity

Intelligent agents and multi-agent technologies are an emerging technology in computing and communications that hold much promise for a wide variety of applications in Information Technology. Agent-based systems range from the simple, single agent system performing tasks such as email filtering, to a very complex, distributed system of multiple agents each involved in individual and system wide goal-oriented activity. With the tremendous growth in the Internet and Internet-based computing and the explosion of commercial activity on the Internet in recent years, intelligent agent-based systems are being applied in a wide variety of electronic commerce applications. In order to be able to act autonomously in a market environment, agents must be able to establish and maintain trust relationships. Without trust, commerce will not take place. This research extends previous work in intelligent agents to include a mechanism for handling the trust relationship and shows how agents can be fully used as intermediaries in commerce.

Sign in / Sign up

Export Citation Format

Share Document