scholarly journals Real-Time Task Assignment Approach Leveraging Reinforcement Learning with Evolution Strategies for Long-Term Latency Minimization in Fog Computing

Sensors ◽  
2018 ◽  
Vol 18 (9) ◽  
pp. 2830 ◽  
Author(s):  
Long Mai ◽  
Nhu-Ngoc Dao ◽  
Minho Park
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Heterogeneous Computing ◽  
Learning Algorithm ◽  
Fog Computing ◽  
Task Assignment ◽  
Evolution Strategies ◽  
Time Task ◽  
Computing Infrastructure

The emerging fog computing technology is characterized by an ultralow latency response, which benefits a massive number of time-sensitive services and applications in the Internet of things (IoT) era. To this end, the fog computing infrastructure must minimize latencies for both service delivery and execution phases. While the transmission latency significantly depends on external factors (e.g., channel bandwidth, communication resources, and interferences), the computation latency can be considered as an internal issue that the fog computing infrastructure could actively self-handle. From this view point, we propose a reinforcement learning approach that utilizes the evolution strategies for real-time task assignment among fog servers to minimize the total computation latency during a long-term period. Experimental results demonstrate that the proposed approach reduces the latency by approximately 16.1% compared to the existing methods. Additionally, the proposed learning algorithm has low computational complexity and an effectively parallel operation; therefore, it is especially appropriate to be implemented in modern heterogeneous computing platforms.

scholarly journals Aircraft Maintenance Check Scheduling Using Reinforcement Learning

Aerospace ◽  
2021 ◽  
Vol 8 (4) ◽  
pp. 113
Author(s):  
Pedro Andrade ◽  
Catarina Silva ◽  
Bernardete Ribeiro ◽  
Bruno F. Santos
Keyword(s):  
Reinforcement Learning ◽  
Time Horizon ◽  
Learning Algorithm ◽  
Initial Conditions ◽  
Q Learning ◽  
Scheduling Policy ◽  
Real Scenario ◽  
Maintenance Plan ◽  
Small Disturbances

This paper presents a Reinforcement Learning (RL) approach to optimize the long-term scheduling of maintenance for an aircraft fleet. The problem considers fleet status, maintenance capacity, and other maintenance constraints to schedule hangar checks for a specified time horizon. The checks are scheduled within an interval, and the goal is to, schedule them as close as possible to their due date. In doing so, the number of checks is reduced, and the fleet availability increases. A Deep Q-learning algorithm is used to optimize the scheduling policy. The model is validated in a real scenario using maintenance data from 45 aircraft. The maintenance plan that is generated with our approach is compared with a previous study, which presented a Dynamic Programming (DP) based approach and airline estimations for the same period. The results show a reduction in the number of checks scheduled, which indicates the potential of RL in solving this problem. The adaptability of RL is also tested by introducing small disturbances in the initial conditions. After training the model with these simulated scenarios, the results show the robustness of the RL approach and its ability to generate efficient maintenance plans in only a few seconds.

Adaptive task scheduling in IoT using reinforcement learning

2020 ◽  
Vol 13 (3) ◽  
pp. 261-282
Author(s):  
Mohammad Khalid Pandit ◽  
Roohie Naaz Mir ◽  
Mohammad Ahsan Chishti
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Real Time ◽  
Resource Utilization ◽  
Task Scheduling ◽  
Clustering Algorithm ◽  
Fog Computing ◽  
Feed Forward Neural Network ◽  
Content Type ◽  
Communication Costs

PurposeThe intelligence in the Internet of Things (IoT) can be embedded by analyzing the huge volumes of data generated by it in an ultralow latency environment. The computational latency incurred by the cloud-only solution can be significantly brought down by the fog computing layer, which offers a computing infrastructure to minimize the latency in service delivery and execution. For this purpose, a task scheduling policy based on reinforcement learning (RL) is developed that can achieve the optimal resource utilization as well as minimum time to execute tasks and significantly reduce the communication costs during distributed execution.Design/methodology/approachTo realize this, the authors proposed a two-level neural network (NN)-based task scheduling system, where the first-level NN (feed-forward neural network/convolutional neural network [FFNN/CNN]) determines whether the data stream could be analyzed (executed) in the resource-constrained environment (edge/fog) or be directly forwarded to the cloud. The second-level NN ( RL module) schedules all the tasks sent by level 1 NN to fog layer, among the available fog devices. This real-time task assignment policy is used to minimize the total computational latency (makespan) as well as communication costs.FindingsExperimental results indicated that the RL technique works better than the computationally infeasible greedy approach for task scheduling and the combination of RL and task clustering algorithm reduces the communication costs significantly.Originality/valueThe proposed algorithm fundamentally solves the problem of task scheduling in real-time fog-based IoT with best resource utilization, minimum makespan and minimum communication cost between the tasks.

scholarly journals Real Time LDR Data Prediction using IoT and Deep Learning Algorithm

2020 ◽  
pp. 158-161
Author(s):  
Chandraprabha S ◽  
Pradeepkumar G ◽  
Dineshkumar Ponnusamy ◽  
Saranya M D ◽  
Satheesh Kumar S ◽  
...  
Keyword(s):  
Deep Learning ◽  
Light Intensity ◽  
Real Time ◽  
Short Term Memory ◽  
Learning Algorithm ◽  
Time Interval ◽  
Deep Learning Algorithm ◽  
Data Prediction ◽  
Solar Plant

This paper outfits artificial intelligence based real time LDR data which is implemented in various applications like indoor lightning, and places where enormous amount of heat is produced, agriculture to increase the crop yield, Solar plant for solar irradiance Tracking. For forecasting the LDR information. The system uses a sensor that can measure the light intensity by means of LDR. The data acquired from sensors are posted in an Adafruit cloud for every two seconds time interval using Node MCU ESP8266 module. The data is also presented on adafruit dashboard for observing sensor variables. A Long short-term memory is used for setting up the deep learning. LSTM module uses the recorded historical data from adafruit cloud which is paired with Node MCU in order to obtain the real-time long-term time series sensor variables that is measured in terms of light intensity. Data is extracted from the cloud for processing the data analytics later the deep learning model is implemented in order to predict future light intensity values.

scholarly journals A weekly, near real-time dataset of the probability of large wildfire across western US forests and woodlands

2018 ◽  
Author(s):  
Miranda E. Gray ◽  
Luke J. Zachmann ◽  
Brett G. Dickson
Keyword(s):  
Real Time ◽  
Learning Algorithm ◽  
Google Earth ◽  
Large Fire ◽  
Digital Repository ◽  
Wildfire Occurrence ◽  
Western Us ◽  
Predicted Probability ◽  
Google Earth Engine

Abstract. There is broad consensus that wildfire activity is likely to increase in western US forests and woodlands over the next century. Therefore, spatial predictions of the potential for large wildfires have immediate and growing relevance to near- and long-term research, planning, and management objectives. Fuels, climate, weather, and the landscape all exert controls on wildfire occurrence and spread, but the dynamics of these controls vary from daily to decadal timescales. Accurate spatial predictions of large wildfires should therefore strive to integrate across these variables and timescales. Here, we describe a high spatial resolution dataset (250-m pixel) of the probability of large wildfire (> 405 ha) across all western US forests and woodlands, from 2005 to the present. The dataset is automatically updated on a weekly basis and in near real-time (i.e., up to the present week) using Google Earth Engine and a "Continuous Integration" pipeline. Each image in the dataset is the output of a machine-learning algorithm, trained on 10 independent, random samples of historic small and large wildfires, and represents the predicted probability of an individual pixel burning in a large fire. This novel workflow is able to integrate the short-term dynamics of fuels and weather into weekly predictions, while also integrating longer-term dynamics of fuels, climate, and the landscape. As a near real-time product, the dataset can provide operational fire managers with immediate, on-the-ground information to closely monitor changing potential for large wildfire occurrence and spread. It can also serve as a foundational dataset for longer-term planning and research, such as strategic targeting of fuels management, fire-smart development at the wildland urban interface, and analysis of trends in wildfire potential over time. Weekly large fire probability GeoTiff products from 2005 through 2017 are archived on Figshare online digital repository with the DOI 10.6084/m9.figshare.5765967 (available at https://doi.org/10.6084/m9.figshare.5765967.v1). Near real-time weekly GeoTiff products and the entire dataset from 2005 on are also continuously uploaded to a Google Cloud Storage bucket at https://console.cloud.google.com/storage/wffr-preds/V1, and also available free of charge with a Google account. Near real-time products and the long-term archive are also available to registered Google Earth Engine (GEE) users as public GEE assets, and can be accessed with the image collection ID "users/mgray/wffr-preds" within GEE.

Operational “Feel” Adjustment by Reinforcement Learning for a Power-Assisted Positioning Task

2009 ◽  
Vol 3 (6) ◽  
pp. 671-680 ◽  
Author(s):  
Tetsuya Morizono ◽  
◽  
Yoji Yamada ◽  
Masatake Higashi ◽  
◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Task Performance ◽  
User Satisfaction ◽  
Learning Algorithm ◽  
Impedance Control ◽  
Experimental Results ◽  
Multiple Goals ◽  
And Task

Controlling “feel” when operating a power-assist robot is important for improving robot operability, user satisfaction, and task performance efficiency. Autonomous adjustment of “feel” is considered with robots under impedance control, and reinforcement learning in adjustment when a task includes repetitive positioning is discussed. Experimental results demonstrate that an operational “feel” pattern appropriate for positioning at a goal is developed by adjustment. Adjustment assuming a single fixed goal is expanded to cases including multiple goals, in which it is assumed that one goal is chosen by a user in real time. To adjust operational “feel” to individual goals, an algorithm infers the goal. The same result as that for a single fixed goal is obtained in experiments, but experimental results suggest that design must be improved to where the accuracy of inference to the goal is taken into account by the adjustment learning algorithm.

Sign in / Sign up

Export Citation Format

Share Document