scholarly journals Exploring Reward Strategies for Wind Turbine Pitch Control by Reinforcement Learning

2020 ◽  
Vol 10 (21) ◽  
pp. 7462
Author(s):  
Jesús Enrique Sierra-García ◽  
Matilde Santos
Keyword(s):  
Reinforcement Learning ◽  
Wind Turbine ◽  
State Estimator ◽  
Pitch Control ◽  
Learning Speed ◽  
Controller Performance ◽  
Greedy Methods ◽  
Relationship Of ◽  
Exploration Exploitation ◽  
The Relationship

In this work, a pitch controller of a wind turbine (WT) inspired by reinforcement learning (RL) is designed and implemented. The control system consists of a state estimator, a reward strategy, a policy table, and a policy update algorithm. Novel reward strategies related to the energy deviation from the rated power are defined. They are designed to improve the efficiency of the WT. Two new categories of reward strategies are proposed: “only positive” (O-P) and “positive-negative” (P-N) rewards. The relationship of these categories with the exploration-exploitation dilemma, the use of ϵ-greedy methods and the learning convergence are also introduced and linked to the WT control problem. In addition, an extensive analysis of the influence of the different rewards in the controller performance and in the learning speed is carried out. The controller is compared with a proportional-integral-derivative (PID) regulator for the same small wind turbine, obtaining better results. The simulations show how the P-N rewards improve the performance of the controller, stabilize the output power around the rated power, and reduce the error over time.

Calculating the aerodynamics of vertical axis wind turbines

2012 ◽  
Vol 34 (3) ◽  
pp. 169-184 ◽  
Author(s):  
Hoang Thi Bich Ngoc
Keyword(s):  
Wind Turbine ◽  
Incidence Angle ◽  
Vertical Axis ◽  
Rotor Blades ◽  
Vertical Axis Wind Turbine ◽  
Horizontal Axis Wind Turbine ◽  
Velocity Triangle ◽  
Relationship Of ◽  
The Relationship

Vertical axis wind turbine technology has been applied last years, very long after horizontal axis wind turbine technology. Aerodynamic problems of vertical axis wind machines are discussible. An important problem is the determination of the incidence law in the interaction between wind and rotor blades. The focus of the work is to establish equations of the incidence depending on the blade azimuth, and to solve them. From these results, aerodynamic torques and power can be calculated. The incidence angle is a parameter of velocity triangle, and both the factors depend not only on the blade azimuth but also on the ratio of rotational speed and horizontal speed. The built computational program allows theoretically selecting the relationship of geometric parameters of wind turbine in accordance with requirements on power, wind speed and installation conditions.

Pupil Diameter Predicts Changes in the Exploration–Exploitation Trade-off: Evidence for the Adaptive Gain Theory

2011 ◽  
Vol 23 (7) ◽  
pp. 1587-1596 ◽  
Author(s):  
Marieke Jepma ◽  
Sander Nieuwenhuis
Keyword(s):  
Reinforcement Learning ◽  
Pupil Diameter ◽  
Behavioral Performance ◽  
Gambling Task ◽  
Trade Off ◽  
Choice Strategy ◽  
Payoff Structure ◽  
Exploration Exploitation ◽  
The Relationship ◽  
Control State

The adaptive regulation of the balance between exploitation and exploration is critical for the optimization of behavioral performance. Animal research and computational modeling have suggested that changes in exploitative versus exploratory control state in response to changes in task utility are mediated by the neuromodulatory locus coeruleus–norepinephrine (LC–NE) system. Recent studies have suggested that utility-driven changes in control state correlate with pupil diameter, and that pupil diameter can be used as an indirect marker of LC activity. We measured participants' pupil diameter while they performed a gambling task with a gradually changing payoff structure. Each choice in this task can be classified as exploitative or exploratory using a computational model of reinforcement learning. We examined the relationship between pupil diameter, task utility, and choice strategy (exploitation vs. exploration), and found that (i) exploratory choices were preceded by a larger baseline pupil diameter than exploitative choices; (ii) individual differences in baseline pupil diameter were predictive of an individual's tendency to explore; and (iii) changes in pupil diameter surrounding the transition between exploitative and exploratory choices correlated with changes in task utility. These findings provide novel evidence that pupil diameter correlates closely with control state, and are consistent with a role for the LC–NE system in the regulation of the exploration–exploitation trade-off in humans.

scholarly journals Unmanned Aerial Vehicle Pitch Control under Delay Using Deep Reinforcement Learning with Continuous Action in Wind Tunnel Test

Aerospace ◽  
2021 ◽  
Vol 8 (9) ◽  
pp. 258
Author(s):  
Daichi Wada ◽  
Sergio A. Araujo-Estrada ◽  
Shane Windsor
Keyword(s):  
Neural Networks ◽  
Reinforcement Learning ◽  
Wind Tunnel ◽  
Real World ◽  
Time Delays ◽  
Wind Tunnel Test ◽  
The Real ◽  
Pitch Control ◽  
Controller Performance ◽  
The Neural Networks

Nonlinear flight controllers for fixed-wing unmanned aerial vehicles (UAVs) can potentially be developed using deep reinforcement learning. However, there is often a reality gap between the simulation models used to train these controllers and the real world. This study experimentally investigated the application of deep reinforcement learning to the pitch control of a UAV in wind tunnel tests, with a particular focus of investigating the effect of time delays on flight controller performance. Multiple neural networks were trained in simulation with different assumed time delays and then wind tunnel tested. The neural networks trained with shorter delays tended to be susceptible to delay in the real tests and produce fluctuating behaviour. The neural networks trained with longer delays behaved more conservatively and did not produce oscillations but suffered steady state errors under some conditions due to unmodeled frictional effects. These results highlight the importance of performing physical experiments to validate controller performance and how the training approach used with reinforcement learning needs to be robust to reality gaps between simulation and the real world.

The relationship of the scleractinian corals to the rugose corals

Paleobiology ◽  
1980 ◽  
Vol 6 (02) ◽  
pp. 146-160 ◽  
Author(s):  
William A. Oliver
Keyword(s):  
Critical Points ◽  
Scleractinian Corals ◽  
Convincing Evidence ◽  
Early Triassic ◽  
Rugose Corals ◽  
History Of ◽  
The Subject ◽  
Relationship Of ◽  
The Relationship ◽  
Biologic Factors

The Mesozoic-Cenozoic coral Order Scleractinia has been suggested to have originated or evolved (1) by direct descent from the Paleozoic Order Rugosa or (2) by the development of a skeleton in members of one of the anemone groups that probably have existed throughout Phanerozoic time. In spite of much work on the subject, advocates of the direct descent hypothesis have failed to find convincing evidence of this relationship. Critical points are:(1) Rugosan septal insertion is serial; Scleractinian insertion is cyclic; no intermediate stages have been demonstrated. Apparent intermediates are Scleractinia having bilateral cyclic insertion or teratological Rugosa.(2) There is convincing evidence that the skeletons of many Rugosa were calcitic and none are known to be or to have been aragonitic. In contrast, the skeletons of all living Scleractinia are aragonitic and there is evidence that fossil Scleractinia were aragonitic also. The mineralogic difference is almost certainly due to intrinsic biologic factors.(3) No early Triassic corals of either group are known. This fact is not compelling (by itself) but is important in connection with points 1 and 2, because, given direct descent, both changes took place during this only stage in the history of the two groups in which there are no known corals.

Skeletal elements of crinoid echinoderms: High-resolution structural and microanalytical results

1983 ◽  
Vol 41 ◽  
pp. 194-195
Author(s):  
D. F. Blake ◽  
L. F. Allard ◽  
D. R. Peacor
Keyword(s):  
High Resolution ◽  
Marine Invertebrates ◽  
Sea Urchins ◽  
Structural Features ◽  
Sea Stars ◽  
High Resolution Tem ◽  
Relationship Of ◽  
The Relationship ◽  
Insight Into

Echinodermata is a phylum of marine invertebrates which has been extant since Cambrian time (c.a. 500 m.y. before the present). Modern examples of echinoderms include sea urchins, sea stars, and sea lilies (crinoids). The endoskeletons of echinoderms are composed of plates or ossicles (Fig. 1) which are with few exceptions, porous, single crystals of high-magnesian calcite. Despite their single crystal nature, fracture surfaces do not exhibit the near-perfect {10.4} cleavage characteristic of inorganic calcite. This paradoxical mix of biogenic and inorganic features has prompted much recent work on echinoderm skeletal crystallography. Furthermore, fossil echinoderm hard parts comprise a volumetrically significant portion of some marine limestones sequences. The ultrastructural and microchemical characterization of modern skeletal material should lend insight into: 1). The nature of the biogenic processes involved, for example, the relationship of Mg heterogeneity to morphological and structural features in modern echinoderm material, and 2). The nature of the diagenetic changes undergone by their ancient, fossilized counterparts. In this study, high resolution TEM (HRTEM), high voltage TEM (HVTEM), and STEM microanalysis are used to characterize tha ultrastructural and microchemical composition of skeletal elements of the modern crinoid Neocrinus blakei.

Studies on Leukemogenic and Sarcomagenic Viruses

1970 ◽  
Vol 28 ◽  
pp. 156-157
Author(s):  
Leon Dmochowski
Keyword(s):  
Electron Microscopy ◽  
Electron Microscope ◽  
Morphological Properties ◽  
Mode Of Development ◽  
Relationship Of ◽  
The Relationship

Electron microscopy has proved to be an invaluable discipline in studies on the relationship of viruses to the origin of leukemia, sarcoma, and other types of tumors in animals and man. The successful cell-free transmission of leukemia and sarcoma in mice, rats, hamsters, and cats, interpreted as due to a virus or viruses, was proved to be due to a virus on the basis of electron microscope studies. These studies demonstrated that all the types of neoplasia in animals of the species examined are produced by a virus of certain characteristic morphological properties similar, if not identical, in the mode of development in all types of neoplasia in animals, as shown in Fig. 1.

Sign in / Sign up

Export Citation Format

Share Document