scholarly journals Reinforcement Learning-Enabled UAV Itinerary Planning for Remote Sensing Applications in Smart Farming

Telecom ◽  
2021 ◽  
Vol 2 (3) ◽  
pp. 255-270
Author(s):  
Saeid Pourroostaei Ardakani ◽  
Ali Cheshmehzangi
Keyword(s):  
Remote Sensing ◽  
Reinforcement Learning ◽  
Data Collection ◽  
Cost Effective ◽  
Environmental Data ◽  
Machine Learning Techniques ◽  
Q Learning ◽  
Sensing Applications ◽  
Learning Technique ◽  
Target Locations

UAV path planning for remote sensing aims to find the best-fitted routes to complete a data collection mission. UAVs plan the routes and move through them to remotely collect environmental data from particular target zones by using sensory devices such as cameras. Route planning may utilize machine learning techniques to autonomously find/select cost-effective and/or best-fitted routes and achieve optimized results including: minimized data collection delay, reduced UAV power consumption, decreased flight traversed distance and maximized number of collected data samples. This paper utilizes a reinforcement learning technique (location and energy-aware Q-learning) to plan UAV routes for remote sensing in smart farms. Through this, the UAV avoids heuristically or blindly moving throughout a farm, but this takes the benefits of environment exploration–exploitation to explore the farm and find the shortest and most cost-effective paths into target locations with interesting data samples to collect. According to the simulation results, utilizing the Q-learning technique increases data collection robustness and reduces UAV resource consumption (e.g., power), traversed paths, and remote sensing latency as compared to two well-known benchmarks, IEMF and TBID, especially if the target locations are dense and crowded in a farm.

scholarly journals Remote Sensing Approaches for Monitoring Mangrove Species, Structure, and Biomass: Opportunities and Challenges

2019 ◽  
Vol 11 (3) ◽  
pp. 230 ◽  
Author(s):  
Tien Pham ◽  
Naoto Yokoya ◽  
Dieu Bui ◽  
Kunihiko Yoshino ◽  
Daniel Friess
Keyword(s):  
Climate Change ◽  
Remote Sensing ◽  
Mangrove Ecosystem ◽  
Carbon Stocks ◽  
Machine Learning Techniques ◽  
Species Structure ◽  
Mangrove Species ◽  
Mitigation And Adaptation ◽  
Sensing Applications ◽  
Wide Range

The mangrove ecosystem plays a vital role in the global carbon cycle, by reducing greenhouse gas emissions and mitigating the impacts of climate change. However, mangroves have been lost worldwide, resulting in substantial carbon stock losses. Additionally, some aspects of the mangrove ecosystem remain poorly characterized compared to other forest ecosystems due to practical difficulties in measuring and monitoring mangrove biomass and their carbon stocks. Without a quantitative method for effectively monitoring biophysical parameters and carbon stocks in mangroves, robust policies and actions for sustainably conserving mangroves in the context of climate change mitigation and adaptation are more difficult. In this context, remote sensing provides an important tool for monitoring mangroves and identifying attributes such as species, biomass, and carbon stocks. A wide range of studies is based on optical imagery (aerial photography, multispectral, and hyperspectral) and synthetic aperture radar (SAR) data. Remote sensing approaches have been proven effective for mapping mangrove species, estimating their biomass, and assessing changes in their extent. This review provides an overview of the techniques that are currently being used to map various attributes of mangroves, summarizes the studies that have been undertaken since 2010 on a variety of remote sensing applications for monitoring mangroves, and addresses the limitations of these studies. We see several key future directions for the potential use of remote sensing techniques combined with machine learning techniques for mapping mangrove areas and species, and evaluating their biomass and carbon stocks.

scholarly journals Using Inverse Reinforcement Learning with Real Trajectories to Get More Trustworthy Pedestrian Simulations

Mathematics ◽  
2020 ◽  
Vol 8 (9) ◽  
pp. 1479
Author(s):  
Francisco Martinez-Gil ◽  
Miguel Lozano ◽  
Ignacio García-Fernández ◽  
Pau Romero ◽  
Dolors Serra ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Value Function ◽  
Machine Learning Techniques ◽  
Inverse Reinforcement Learning ◽  
The Real ◽  
Q Learning ◽  
Learning Framework ◽  
Entropy Principle ◽  
Real Behavior ◽  
Function Approximator

Reinforcement learning is one of the most promising machine learning techniques to get intelligent behaviors for embodied agents in simulations. The output of the classic Temporal Difference family of Reinforcement Learning algorithms adopts the form of a value function expressed as a numeric table or a function approximator. The learned behavior is then derived using a greedy policy with respect to this value function. Nevertheless, sometimes the learned policy does not meet expectations, and the task of authoring is difficult and unsafe because the modification of one value or parameter in the learned value function has unpredictable consequences in the space of the policies it represents. This invalidates direct manipulation of the learned value function as a method to modify the derived behaviors. In this paper, we propose the use of Inverse Reinforcement Learning to incorporate real behavior traces in the learning process to shape the learned behaviors, thus increasing their trustworthiness (in terms of conformance to reality). To do so, we adapt the Inverse Reinforcement Learning framework to the navigation problem domain. Specifically, we use Soft Q-learning, an algorithm based on the maximum causal entropy principle, with MARL-Ped (a Reinforcement Learning-based pedestrian simulator) to include information from trajectories of real pedestrians in the process of learning how to navigate inside a virtual 3D space that represents the real environment. A comparison with the behaviors learned using a Reinforcement Learning classic algorithm (Sarsa(λ)) shows that the Inverse Reinforcement Learning behaviors adjust significantly better to the real trajectories.

scholarly journals Parallel Implementation of Reinforcement Learning Q-Learning Technique for FPGA

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 2782-2798 ◽  
Author(s):  
Lucileide M. D. Da Silva ◽  
Matheus F. Torquato ◽  
Marcelo A. C. Fernandes
Keyword(s):  
Reinforcement Learning ◽  
Parallel Implementation ◽  
Q Learning ◽  
Learning Technique

scholarly journals Q-Learning based Routing Protocol to Enhance Network Lifetime in WSNs

2021 ◽  
Vol 13 (2) ◽  
pp. 57-80
Author(s):  
Arunita Kundaliya ◽  
D.K. Lobiyal
Keyword(s):  
Reinforcement Learning ◽  
Network Lifetime ◽  
Residual Energy ◽  
Efficient Solutions ◽  
Machine Learning Techniques ◽  
Q Learning ◽  
Learning Techniques ◽  
Aodv Protocol ◽  
Optimal Action ◽  
Additional Memory

In resource constraint Wireless Sensor Networks (WSNs), enhancement of network lifetime has been one of the significantly challenging issues for the researchers. Researchers have been exploiting machine learning techniques, in particular reinforcement learning, to achieve efficient solutions in the domain of WSN. The objective of this paper is to apply Q-learning, a reinforcement learning technique, to enhance the lifetime of the network, by developing distributed routing protocols. Q-learning is an attractive choice for routing due to its low computational requirements and additional memory demands. To facilitate an agent running at each node to take an optimal action, the approach considers node’s residual energy, hop length to sink and transmission power. The parameters, residual energy and hop length, are used to calculate the Q-value, which in turn is used to decide the optimal next-hop for routing. The proposed protocols’ performance is evaluated through NS3 simulations, and compared with AODV protocol in terms of network lifetime, throughput and end-to-end delay.

Cost-Effective Air Quality Monitoring System Based on an Open-Source Electronics Platform for Three-Dimensional Atmospheric Environmental Data Collection

2020 ◽  
Author(s):  
Yi-Chung Tung ◽  
Dao-Ming Chang ◽  
Chuang-Yuan Kuo
Keyword(s):  
Air Quality ◽  
Data Collection ◽  
Open Source ◽  
Monitoring System ◽  
Three Dimensional ◽  
Cost Effective ◽  
Environmental Data ◽  
Quality Monitoring ◽  
Air Quality Monitoring ◽  
Spatiotemporal Resolution

<p>Air pollution and extreme weather patterns have become serious issues over the world, especially in highly urbanized areas.  In order to detailed study the atmospheric environmental change, the capability to perform high spatiotemporal resolution atmospheric environmental data collection is highly desired.  In this research, we develop a cost-effective air quality monitoring system based on as open-source electronics platform (Arduino Uno Rev3) with multiple environmental sensing modules including particulate matter (PM) concentration, temperature, humidity, and sound sensors.  An integrated monitoring system with one weather station (precipitation and wind sensors) and two sets of environmental sensors set up in different heights from the ground costs less than USD$300.  The entire system is powered by a battery for portability, and all the data can be stored in a secure digital (SD) memory card for long-term monitoring. The cost-effectiveness makes it feasible for large-scale field tests with three-dimensional (3D) spatial resolution.  In the experiments, the system is tested in urban areas, and the data collection performance has been confirmed.  The results show that the data with single minute resolution can be successfully achieved in real-world scenarios with high air temperature (> 38<sup>o</sup>C) and rain conditions for more than 65 hours with a single-time battery setup.  In addition, the data collected from different heights have shown distinct atmospheric environmental patterns suggesting that it is critical to perform 3D high spatiotemporal measurement and modeling for city-scale studies.</p>

scholarly journals A Review of Remote Sensing Approaches for Monitoring Blue Carbon Ecosystems: Mangroves, Seagrassesand Salt Marshes during 2010–2018

Sensors ◽  
2019 ◽  
Vol 19 (8) ◽  
pp. 1933 ◽  
Author(s):  
Tien Dat Pham ◽  
Junshi Xia ◽  
Nam Thang Ha ◽  
Dieu Tien Bui ◽  
Nga Nhu Le ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Salt Marshes ◽  
Vital Role ◽  
Blue Carbon ◽  
Aerial Photographs ◽  
Machine Learning Techniques ◽  
Coastal Vegetation ◽  
Multispectral Data ◽  
Sensing Applications ◽  
Optical Imagery

Blue carbon (BC) ecosystems are an important coastal resource, as they provide a range of goods and services to the environment. They play a vital role in the global carbon cycle by reducing greenhouse gas emissions and mitigating the impacts of climate change. However, there has been a large reduction in the global BC ecosystems due to their conversion to agriculture and aquaculture, overexploitation, and removal for human settlements. Effectively monitoring BC ecosystems at large scales remains a challenge owing to practical difficulties in monitoring and the time-consuming field measurement approaches used. As a result, sensible policies and actions for the sustainability and conservation of BC ecosystems can be hard to implement. In this context, remote sensing provides a useful tool for mapping and monitoring BC ecosystems faster and at larger scales. Numerous studies have been carried out on various sensors based on optical imagery, synthetic aperture radar (SAR), light detection and ranging (LiDAR), aerial photographs (APs), and multispectral data. Remote sensing-based approaches have been proven effective for mapping and monitoring BC ecosystems by a large number of studies. However, to the best of our knowledge, this is the first comprehensive review on the applications of remote sensing techniques for mapping and monitoring BC ecosystems. The main goal of this review is to provide an overview and summary of the key studies undertaken from 2010 onwards on remote sensing applications for mapping and monitoring BC ecosystems. Our review showed that optical imagery, such as multispectral and hyper-spectral data, is the most common for mapping BC ecosystems, while the Landsat time-series are the most widely-used data for monitoring their changes on larger scales. We investigate the limitations of current studies and suggest several key aspects for future applications of remote sensing combined with state-of-the-art machine learning techniques for mapping coastal vegetation and monitoring their extents and changes.

scholarly journals ANALYSIS OF THE APPLICATION OF REINFORCEMENT LEARNING ALGORITHMS ON THE STARCRAFT II VIDEO GAME

2020 ◽  
Vol 11 (4) ◽  
Author(s):  
Leandro Vian ◽  
Marcelo De Gomensoro Malheiros
Keyword(s):  
Reinforcement Learning ◽  
Video Game ◽  
Cost Effective ◽  
Machine Learning Techniques ◽  
Specific Training ◽  
Board Games ◽  
Learning Techniques ◽  
Technical Issues ◽  
Strategy Game ◽  
Real Time Strategy Game

In recent years Machine Learning techniques have become the driving force behind the worldwide emergence of Artificial Intelligence, producing cost-effective and precise tools for pattern recognition and data analysis. A particular approach for the training of neural networks, Reinforcement Learning (RL), achieved prominence creating almost unbeatable artificial opponents in board games like Chess or Go, and also on video games. This paper gives an overview of Reinforcement Learning and tests this approach against a very popular real-time strategy game, Starcraft II. Our goal is to examine the tools and algorithms readily available for RL, also addressing different scenarios where a neural network can be linked to Starcraft II to learn by itself. This work describes both the technical issues involved and the preliminary results obtained by the application of two specific training strategies, A2C and DQN.

An Analysis of Rule Deletion Scheme in XCS on Reinforcement Learning Problem

2017 ◽  
Vol 21 (5) ◽  
pp. 876-884
Author(s):  
Masaya Nakata ◽  
◽  
Tomoki Hamagami
Keyword(s):  
Reinforcement Learning ◽  
Learning Problem ◽  
Rule Based ◽  
Learning Mechanism ◽  
Q Learning ◽  
State Action ◽  
Classifier System ◽  
Specific Subset ◽  
Learning Technique

The XCS classifier system is an evolutionary rule-based learning technique powered by a Q-learning like learning mechanism. It employs a global deletion scheme to delete rules from all rules covering all state-action pairs. However, the optimality of this scheme remains unclear owing to the lack of intensive analysis. We here introduce two deletion schemes: 1) local deletion, which can be applied to a subset of rules covering each state (a match set), and 2) stronger local deletion, which can be applied to a more specific subset covering each state-action pair (an action set). The aim of this paper is to reveal how the above three deletion schemes affect the performance of XCS. Our analysis shows that the local deletion schemes promote the elimination of inaccurate rules compared with the global deletion scheme. However, the stronger local deletion scheme occasionally deletes a good rule. We further show that the two local deletion schemes greatly improve the performance of XCS on a set of noisy maze problems. Although the localization strength of the proposed deletion schemes may require consideration, they can be adequate for XCS rather than the original global deletion scheme.

Sign in / Sign up

Export Citation Format

Share Document