A Decentralized Economic Load-Balancing Framework among Federated Cloudlets

Edge computing servers like cloudlets from different service providers compensate scarce computational and storage resources of mobile devices, are distributed across access networks. However, the dynamically varying computational requirements of associated mobile devices make cloudlets either overloaded or under-loaded. Hence, load balancing among neighboring cloudlets appears to be an essential research problem. Especially, the load balancing problem among federated cloudlets from the same as well as different service providers for low-latency applications needs significant attention. Thus, in this paper, we propose a decentralized load balancing framework among federated cloudlets for low-latency applications that focuses on latency bound rather than latency minimization. In this framework, we employ dynamic processor slicing for handling heterogeneous classes of job requests. We propose a continuous-action reinforcement learning automata-based algorithm that enables cloudlets to independently compute the load balancing strategies in a completely distributed network setting without any exhaustive control message exchange. To capture the economic interaction among federated cloudlets, we model this load balancing problem as an economic and non-cooperative game and by scaffolding the properties of the game formulation, we achieve faster convergence of the reinforcement learning automata. Furthermore, through extensive simulations, we study the impacts of exploration and exploitation on learning accuracy.

A Decentralized Economic Load-Balancing Framework among Federated Cloudlets

Edge computing servers like cloudlets from different service providers compensate scarce computational and storage resources of mobile devices, are distributed across access networks. However, the dynamically varying computational requirements of associated mobile devices make cloudlets either overloaded or under-loaded. Hence, load balancing among neighboring cloudlets appears to be an essential research problem. Especially, the load balancing problem among federated cloudlets from the same as well as different service providers for low-latency applications needs significant attention. Thus, in this paper, we propose a decentralized load balancing framework among federated cloudlets for low-latency applications that focuses on latency bound rather than latency minimization. In this framework, we employ dynamic processor slicing for handling heterogeneous classes of job requests. We propose a continuous-action reinforcement learning automata-based algorithm that enables cloudlets to independently compute the load balancing strategies in a completely distributed network setting without any exhaustive control message exchange. To capture the economic interaction among federated cloudlets, we model this load balancing problem as an economic and non-cooperative game and by scaffolding the properties of the game formulation, we achieve faster convergence of the reinforcement learning automata. Furthermore, through extensive simulations, we study the impacts of exploration and exploitation on learning accuracy.

Multivariable System Identification Method Based on Continuous Action Reinforcement Learning Automata

In this work, a closed-loop identification method based on a reinforcement learning algorithm is proposed for multiple-input multiple-output (MIMO) systems. This method could be an attractive alternative solution to the problem that the current frequency-domain identification algorithms are usually dependent on the attenuation factor. With this method, after continuously interacting with the environment, the optimal attenuation factor can be identified by the continuous action reinforcement learning automata (CARLA), and then the corresponding parameters could be estimated in the end. Moreover, the proposed method could be applied to time-varying systems online due to its online learning ability. The simulation results suggest that the presented approach can meet the requirement of identification accuracy in both square and non-square systems.

IDS Using Reinforcement Learning Automata for Preserving Security in Cloud Environment

Cloud computing relies on sharing computing resources. With high availability and accessibility of resources, cloud computing is under the threat of major cyber-attacks. To detect attacks and preserve security in cloud environment, having an efficient intrusion detection system (IDS) is required. In this article, an effective and efficient IDS is proposed to maintain high level security of data in cloud. The authors have incorporated Reinforcement Learning Automata with their proposed IDS while detecting and classifying attacks. Using learning automata an effective rule set is generated with the proposed algorithm from vast training set to improve the learning process at reduced computation cost and time. After which, the proposed reinforcement learning algorithm helps in classification of attacks accurately using the reinforcement signal. This proposed model was experimented with NSL-KDD as well as KDD 10% dataset and have proved its robustness by detecting attacks more accurately being an IDS.