Meteorological Data Forecast using RNN

2017 ◽  
Vol 9 (1) ◽  
pp. 61-74 ◽  
Author(s):  
Stefan Balluff ◽  
Jörg Bendfeld ◽  
Stefan Krauter
Keyword(s):  
Neural Networks ◽  
Wind Speed ◽  
Linear Prediction ◽  
Learning Algorithm ◽  
Meteorological Data ◽  
System Modeling ◽  
Electrical Power ◽  
Descent Method ◽  
Gradient Descent Method ◽  
Earth System Modeling

Gathering knowledge not only of the current but also the upcoming wind speed is getting more and more important as the experience of operating and maintaining wind turbines is increasing. Not only with regards to operation and maintenance tasks such as gearbox and generator checks but moreover due to the fact that energy providers have to sell the right amount of their converted energy at the European energy markets, the knowledge of the wind and hence electrical power of the next day is of key importance. Selling more energy as has been offered is penalized as well as offering less energy as contractually promised. In addition to that the price per offered kWh decreases in case of a surplus of energy. Achieving a forecast there are various methods in computer science: fuzzy logic, linear prediction or neural networks. This paper presents current results of wind speed forecasts using recurrent neural networks (RNN) and the gradient descent method plus a backpropagation learning algorithm. Data used has been extracted from NASA's Modern Era-Retrospective analysis for Research and Applications (MERRA) which is calculated by a GEOS-5 Earth System Modeling and Data Assimilation system. The presented results show that wind speed data can be forecasted using historical data for training the RNN. Nevertheless, the current set up system lacks robustness and can be improved further with regards to accuracy.

Meteorological Data Forecast using RNN

2020 ◽  
pp. 905-920
Author(s):  
Stefan Balluff ◽  
Jörg Bendfeld ◽  
Stefan Krauter
Keyword(s):  
Neural Networks ◽  
Wind Speed ◽  
Linear Prediction ◽  
Learning Algorithm ◽  
Meteorological Data ◽  
System Modeling ◽  
Descent Method ◽  
Gradient Descent Method ◽  
Earth System Modeling ◽  
Set Up

Gathering knowledge not only of the current but also the upcoming wind speed is getting more and more important as the experience of operating and maintaining wind turbines is increasing. Not only with regards to operation and maintenance tasks such as gearbox and generator checks but moreover due to the fact that energy providers have to sell the right amount of their converted energy at the European energy markets, the knowledge of the wind and hence electrical power of the next day is of key importance. Selling more energy as has been offered is penalized as well as offering less energy as contractually promised. In addition to that the price per offered kWh decreases in case of a surplus of energy. Achieving a forecast there are various methods in computer science: fuzzy logic, linear prediction or neural networks. This paper presents current results of wind speed forecasts using recurrent neural networks (RNN) and the gradient descent method plus a backpropagation learning algorithm. Data used has been extracted from NASA's Modern Era-Retrospective analysis for Research and Applications (MERRA) which is calculated by a GEOS-5 Earth System Modeling and Data Assimilation system. The presented results show that wind speed data can be forecasted using historical data for training the RNN. Nevertheless, the current set up system lacks robustness and can be improved further with regards to accuracy.

scholarly journals Adaptive Natural Gradient Method for Learning of Stochastic Neural Networks in Mini-Batch Mode

2019 ◽  
Vol 9 (21) ◽  
pp. 4568
Author(s):  
Hyeyoung Park ◽  
Kwanyong Lee
Keyword(s):  
Neural Networks ◽  
Gradient Descent ◽  
Learning Algorithm ◽  
Descent Method ◽  
Benchmark Problems ◽  
Stochastic Neural Networks ◽  
Gradient Descent Method ◽  
Natural Gradient ◽  
Convergence Properties ◽  
Data Set

Gradient descent method is an essential algorithm for learning of neural networks. Among diverse variations of gradient descent method that have been developed for accelerating learning speed, the natural gradient learning is based on the theory of information geometry on stochastic neuromanifold, and is known to have ideal convergence properties. Despite its theoretical advantages, the pure natural gradient has some limitations that prevent its practical usage. In order to get the explicit value of the natural gradient, it is required to know true probability distribution of input variables, and to calculate inverse of a matrix with the square size of the number of parameters. Though an adaptive estimation of the natural gradient has been proposed as a solution, it was originally developed for online learning mode, which is computationally inefficient for the learning of large data set. In this paper, we propose a novel adaptive natural gradient estimation for mini-batch learning mode, which is commonly adopted for big data analysis. For two representative stochastic neural network models, we present explicit rules of parameter updates and learning algorithm. Through experiments on three benchmark problems, we confirm that the proposed method has superior convergence properties to the conventional methods.

scholarly journals SOME METHODS OF ADAPTIVE MULTILAYER NEURAL NETWORKS TRAINING

2014 ◽  
pp. 99-106
Author(s):  
Leonid Makhnist ◽  
Nikolaj Maniakov ◽  
Nikolaj Maniakov
Keyword(s):  
Neural Networks ◽  
Basic Concept ◽  
Gradient Descent ◽  
Descent Method ◽  
Gradient Descent Method ◽  
New Techniques ◽  
Adaptive Training ◽  
Multilayer Neural Networks

Is proposed two new techniques for multilayer neural networks training. Its basic concept is based on the gradient descent method. For every methodic are showed formulas for calculation of the adaptive training steps. Presented matrix algorithmizations for all of these techniques are very helpful in its program realization.

Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information

1997 ◽  
Vol 9 (7) ◽  
pp. 1457-1482 ◽  
Author(s):  
Howard Hua Yang ◽  
Shun-ichi Amari
Keyword(s):  
Mutual Information ◽  
Maximum Entropy ◽  
Learning Algorithm ◽  
Adaptive Method ◽  
Descent Method ◽  
Stochastic Gradient Descent ◽  
Gradient Descent Method ◽  
Natural Gradient ◽  
Blind Separation ◽  
Efficient Learning

There are two major approaches for blind separation: maximum entropy (ME) and minimum mutual information (MMI). Both can be implemented by the stochastic gradient descent method for obtaining the demixing matrix. The MI is the contrast function for blind separation; the entropy is not. To justify the ME, the relation between ME and MMI is first elucidated by calculating the first derivative of the entropy and proving that the mean subtraction is necessary in applying the ME and at the solution points determined by the MI, the ME will not update the demixing matrix in the directions of increasing the cross-talking. Second, the natural gradient instead of the ordinary gradient is introduced to obtain efficient algorithms, because the parameter space is a Riemannian space consisting of matrices. The mutual information is calculated by applying the Gram-Charlier expansion to approximate probability density functions of the outputs. Finally, we propose an efficient learning algorithm that incorporates with an adaptive method of estimating the unknown cumulants. It is shown by computer simulation that the convergence of the stochastic descent algorithms is improved by using the natural gradient and the adaptively estimated cumulants.

THE APPLICATION OF FEEDFORWARD NEURAL NETWORKS IN VLSI FABRICATION PROCESS OPTIMIZATION

2001 ◽  
Vol 01 (01) ◽  
pp. 83-90 ◽  
Author(s):  
WANG XIANGDONG ◽  
WANG SHOUJUE
Keyword(s):  
Neural Networks ◽  
Manufacturing Process ◽  
Large Scale ◽  
Maximum Yield ◽  
Feedforward Neural Networks ◽  
Descent Method ◽  
Gradient Descent Method ◽  
Manufacturing Process Control ◽  
Wafer Probing ◽  
Process Learning

In this paper, we present a neural-based manufacturing process control system for semiconductor factories to improve the die yield. A model based on neural networks is proposed to simulate Very Large-Scale Integrated (VLSI) manufacturing process. Learning from the historical processing lists with Radial Basis Function (RBF), we simulate the functional relationship between the wafer probing parameters and the die yield. Then we use a gradient-descent method to search a set of 'optimal' parameters that lead to the maximum yield of the model. At last, we adjust the specification in the practical semiconductor manufacturing process. The average die yield increased from 51.7% to 57.5% after the system had been applied in Huajing Corporation.

scholarly journals Canoeing Motion Tracking and Analysis via Multi-Sensors Fusion

Sensors ◽  
2020 ◽  
Vol 20 (7) ◽  
pp. 2110 ◽  
Author(s):  
Long Liu ◽  
Sen Qiu ◽  
ZheLong Wang ◽  
Jie Li ◽  
JiaXin Wang
Keyword(s):  
Motion Tracking ◽  
Learning Algorithm ◽  
Inertial Sensor ◽  
Recovery Phase ◽  
Descent Method ◽  
Sensor Nodes ◽  
Sensor Calibration ◽  
Gradient Descent Method ◽  
Whole Process ◽  
Training Methodologies

Coaches and athletes are constantly seeking novel training methodologies in an attempt to improve athletic performance. This paper proposes a method of rowing sport capture and analysis based on Inertial Measurement Units (IMUs). A canoeist’s motion was collected by multiple miniature inertial sensor nodes. The gradient descent method was used to fuse data and obtain the canoeist’s attitude information after sensor calibration, and then the motions of canoeist’s actions were reconstructed. Stroke quality was performed based on the estimated joint angles. Machine learning algorithm was used as the classification method to divide the stroke cycle into different phases, including propulsion-phase and recovery-phase, a quantitative kinematic analysis was carried out. Experiments conducted in this paper demonstrated that our method possesses the capacity to reveal the similarities and differences between novice and coach, the whole process of canoeist’s motions can be analyzed with satisfactory accuracy validated by videography method. It can provide quantitative data for coaches or athletes, which can be used to improve the skills of rowers.

Classification and Prediction with Neural Networks

2011 ◽  
pp. 76-107
Author(s):  
Arnošt Veselý
Keyword(s):  
Neural Networks ◽  
Error Function ◽  
Descent Method ◽  
Cross Entropy ◽  
Gradient Descent Method ◽  
Classification Problems ◽  
Network Training ◽  
The Neural Network ◽  
Gradient Calculation ◽  
Entropy Error

This chapter deals with applications of artificial neural networks in classification and regression problems. Based on theoretical analysis it demonstrates that in classification problems one should use cross-entropy error function rather than the usual sum-of-square error function. Using gradient descent method for finding the minimum of the cross entropy error function, leads to the well-known backpropagation of error scheme of gradient calculation if at the output layer of the neural network the neurons with logistic or softmax output functions are used. The author believes that understanding the underlying theory presented in this chapter will help researchers in medical informatics to choose more suitable network architectures for medical applications and that it helps them to carry out the network training more effectively.

scholarly journals A Pre-Trained Fuzzy Reinforcement Learning Method for the Pursuing Satellite in a One-to-One Game in Space

Sensors ◽  
2020 ◽  
Vol 20 (8) ◽  
pp. 2253
Author(s):  
Xiao Wang ◽  
Peng Shi ◽  
Yushan Zhao ◽  
Yue Sun
Keyword(s):  
Reinforcement Learning ◽  
Gradient Descent ◽  
Fuzzy Inference ◽  
Learning Algorithm ◽  
Control Policy ◽  
Descent Method ◽  
Gradient Descent Method ◽  
One To One ◽  
Inference Systems ◽  
First Time

In order to help the pursuer find its advantaged control policy in a one-to-one game in space, this paper proposes an innovative pre-trained fuzzy reinforcement learning algorithm, which is conducted in the x, y, and z channels separately. Compared with the previous algorithms applied in ground games, this is the first time reinforcement learning has been introduced to help the pursuer in space optimize its control policy. The known part of the environment is utilized to help the pursuer pre-train its consequent set before learning. An actor-critic framework is built in each moving channel of the pursuer. The consequent set of the pursuer is updated through the gradient descent method in fuzzy inference systems. The numerical experimental results validate the effectiveness of the proposed algorithm in improving the game ability of the pursuer.

A Novel Interactively Recurrent Self-Evolving Fuzzy CMAC and Its Classification Applications

2015 ◽  
Vol 14 (03) ◽  
pp. 1550019
Author(s):  
Jyun-Guo Wang ◽  
Shen-Chuan Tai ◽  
Cheng-Jian Lin
Keyword(s):  
Structure Learning ◽  
Learning Algorithm ◽  
Descent Method ◽  
Superior Performance ◽  
Gradient Descent Method ◽  
Parameter Learning ◽  
Cerebellar Model Articulation Controller ◽  
Classification Problems ◽  
Fuzzy Cmac ◽  
Firing Strength

In this paper, an Interactively Recurrent Self-evolving Fuzzy Cerebellar Model Articulation Controller (IRSFCMAC) model is developed for solving classification problems. The proposed IRSFCMAC classifier consists of internal feedback and external loops, which are generated by the hypercube cell firing strength to itself and other hypercube cells. The learning process of the IRSFCMAC gets started with an empty hypercube base, and then all of hypercube cells are generated and learned online via structure and parameter learning, respectively. The structure learning algorithm is based on the degree measure to determine the number of hypercube cells. The parameter learning algorithm, based on the gradient descent method, adjusts the shapes of the membership functions and the corresponding fuzzy weights of the IRSFCMAC. Finally, the proposed IRSFCMAC model is tested by four benchmark classification problems. Experimental results show that the proposed IRSFCMAC model has superior performance than traditional FCMAC and other models.

Sign in / Sign up

Export Citation Format

Share Document