scholarly journals Controlling colloidal crystals via morphing energy landscapes and reinforcement learning

2020 ◽  
Vol 6 (48) ◽  
pp. eabd6716
Author(s):  
Jianli Zhang ◽  
Junyan Yang ◽  
Yuanxing Zhang ◽  
Michael A. Bevan
Keyword(s):  
Reinforcement Learning ◽  
Electric Fields ◽  
Large Scale ◽  
Colloidal Particles ◽  
Control Method ◽  
Relaxation Times ◽  
Colloidal Crystals ◽  
Hierarchical Structures ◽  
Control Policy ◽  
Energy Landscapes

We report a feedback control method to remove grain boundaries and produce circular shaped colloidal crystals using morphing energy landscapes and reinforcement learning–based policies. We demonstrate this approach in optical microscopy and computer simulation experiments for colloidal particles in ac electric fields. First, we discover how tunable energy landscape shapes and orientations enhance grain boundary motion and crystal morphology relaxation. Next, reinforcement learning is used to develop an optimized control policy to actuate morphing energy landscapes to produce defect-free crystals orders of magnitude faster than natural relaxation times. Morphing energy landscapes mechanistically enable rapid crystal repair via anisotropic stresses to control defect and shape relaxation without melting. This method is scalable for up to at least N = 103 particles with mean process times scaling as N0.5. Further scalability is possible by controlling parallel local energy landscapes (e.g., periodic landscapes) to generate large-scale global defect-free hierarchical structures.

scholarly journals End-to-End Deep Reinforcement Learning for Image-Based UAV Autonomous Control

2021 ◽  
Vol 11 (18) ◽  
pp. 8419
Author(s):  
Jiang Zhao ◽  
Jiaming Sun ◽  
Zhihao Cai ◽  
Longhong Wang ◽  
Yingxun Wang
Keyword(s):  
Reinforcement Learning ◽  
Network Architecture ◽  
Control Method ◽  
Control Policy ◽  
Input Image ◽  
Autonomous Control ◽  
Policy Network ◽  
Model Free ◽  
Control Command ◽  
End To End

To achieve the perception-based autonomous control of UAVs, schemes with onboard sensing and computing are popular in state-of-the-art work, which often consist of several separated modules with respective complicated algorithms. Most methods depend on handcrafted designs and prior models with little capacity for adaptation and generalization. Inspired by the research on deep reinforcement learning, this paper proposes a new end-to-end autonomous control method to simplify the separate modules in the traditional control pipeline into a single neural network. An image-based reinforcement learning framework is established, depending on the design of the network architecture and the reward function. Training is performed with model-free algorithms developed according to the specific mission, and the control policy network can map the input image directly to the continuous actuator control command. A simulation environment for the scenario of UAV landing was built. In addition, the results under different typical cases, including both the small and large initial lateral or heading angle offsets, show that the proposed end-to-end method is feasible for perception-based autonomous control.

scholarly journals Control Method of Buses and Lines Using Reinforcement Learning for Short Circuit Current Reduction

2020 ◽  
Vol 12 (22) ◽  
pp. 9333
Author(s):  
Sangwook Han
Keyword(s):  
Reinforcement Learning ◽  
Power Systems ◽  
Large Scale ◽  
Control Method ◽  
Short Circuit ◽  
Power Grid ◽  
Short Circuit Current ◽  
Control Methods ◽  
Grid Operation ◽  
Short Circuit Currents

This paper proposes a reinforcement learning-based approach that optimises bus and line control methods to solve the problem of short circuit currents in power systems. Expansion of power grids leads to concentrated power output and more lines for large-scale transmission, thereby increasing short circuit currents. The short circuit currents must be managed systematically by controlling the buses and lines such as separating, merging, and moving a bus, line, or transformer. However, there are countless possible control schemes in an actual grid. Moreover, to ensure compliance with power system reliability standards, no bus should exceed breaker capacity nor should lines or transformers be overloaded. For this reason, examining and selecting a plan requires extensive time and effort. To solve these problems, this paper introduces reinforcement learning to optimise control methods. By providing appropriate rewards for each control action, a policy is set, and the optimal control method is obtained through a maximising value method. In addition, a technique is presented that systematically defines the bus and line separation measures, limits the range of measures to those with actual power grid applicability, and reduces the optimisation time while increasing the convergence probability and enabling use in actual power grid operation. In the future, this technique will contribute significantly to establishing power grid operation plans based on short circuit currents.

Electrical charging of hail pellets in a polarizing electric held

1962 ◽  
Vol 266 (1326) ◽  
pp. 387-401 ◽  
Keyword(s):  
Surface Temperature ◽  
Electric Fields ◽  
Particle Shape ◽  
Large Scale ◽  
Relaxation Times ◽  
Ice Crystals ◽  
Theoretical Equation ◽  
The Times ◽  
Electrical Charging

Earlier experiments in which the authors investigated the electrical charging of a simulated hailstone by collisions with ice crystals, and by the accretion, freezing and bursting of supercooled droplets, have now been repeated in the presence of polarizing electric fields of up to about 1000 Vcm -1 , which are typical of large-scale fields in thunderstorms. It is found that such fields have no detectable influence on the charging produced by rebounding ice crystals, apparently because the times of contact are less than the relaxation times for the conduction of charge between the particles. After much longer times the charge A q transferred between two ice spheres of radius R, r in a uniform polarizing field E agrees with the theoretical equation Δ q = {γ± E cos θ ={γ2( q/R 2 )} r 2 , where γ 1 and γ 2 are calculable functions of r/R , q is the charge on the larger sphere and θ is the angle between the line of centres and the field. The dependence upon time of contact, particle shape and surface temperature is also investigated. The charging which accompanies the impaction and freezing of supercooled droplets on the hailstone is altered by only about 10 % by the application of fields of order 1000Vcm -1 . The conclusion is that the charging of hailstones by either process will not be greatly accelerated by the build-up of polarizing fields in thunderstorms.

scholarly journals Coordinated Control of Distributed Traffic Signal Based on Multiagent Cooperative Game

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Zhenghua Zhang ◽  
Jin Qian ◽  
Chongxin Fang ◽  
Guoshu Liu ◽  
Quan Su
Keyword(s):  
Reinforcement Learning ◽  
Cooperative Game ◽  
Large Scale ◽  
Control Method ◽  
Transportation Network ◽  
Traffic Signal ◽  
Learning Ability ◽  
Hybrid Network ◽  
Traffic Signal Control ◽  
Communication And Coordination

In the adaptive traffic signal control (ATSC), reinforcement learning (RL) is a frontier research hotspot, combined with deep neural networks to further enhance its learning ability. The distributed multiagent RL (MARL) can avoid this kind of problem by observing some areas of each local RL in the complex plane traffic area. However, due to the limited communication capabilities between each agent, the environment becomes partially visible. This paper proposes multiagent reinforcement learning based on cooperative game (CG-MARL) to design the intersection as an agent structure. The method considers not only the communication and coordination between agents but also the game between agents. Each agent observes its own area to learn the RL strategy and value function, then concentrates the Q function from different agents through a hybrid network, and finally forms its own final Q function in the entire large-scale transportation network. The results show that the proposed method is superior to the traditional control method.

Integrating human experience in deep reinforcement learning for multi-UAV collision detection and avoidance

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Guanzheng Wang ◽  
Yinbo Xu ◽  
Zhihong Liu ◽  
Xin Xu ◽  
Xiangke Wang ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Collision Detection ◽  
Large Scale ◽  
Control Method ◽  
Hybrid Control ◽  
Human Experience ◽  
Content Type ◽  
Aerial Vehicle ◽  
Speed Up ◽  
Multi Uav

Purpose This paper aims to realize a fully distributed multi-UAV collision detection and avoidance based on deep reinforcement learning (DRL). To deal with the problem of low sample efficiency in DRL and speed up the training. To improve the applicability and reliability of the DRL-based approach in multi-UAV control problems. Design/methodology/approach In this paper, a fully distributed collision detection and avoidance approach for multi-UAV based on DRL is proposed. A method that integrates human experience into policy training via a human experience-based adviser is proposed. The authors propose a hybrid control method which combines the learning-based policy with traditional model-based control. Extensive experiments including simulations, real flights and comparative experiments are conducted to evaluate the performance of the approach. Findings A fully distributed multi-UAV collision detection and avoidance method based on DRL is realized. The reward curve shows that the training process when integrating human experience is significantly accelerated and the mean episode reward is higher than the pure DRL method. The experimental results show that the DRL method with human experience integration has a significant improvement than the pure DRL method for multi-UAV collision detection and avoidance. Moreover, the safer flight brought by the hybrid control method has also been validated. Originality/value The fully distributed architecture is suitable for large-scale unmanned aerial vehicle (UAV) swarms and real applications. The DRL method with human experience integration has significantly accelerated the training compared to the pure DRL method. The proposed hybrid control strategy makes up for the shortcomings of two-dimensional light detection and ranging and other puzzles in applications.

Solid Lipid Nanoparticles: A Promising Drug Delivery Technology

2009 ◽  
Vol 2 (2) ◽  
pp. 509-516
Author(s):  
S. Pragati ◽  
S. Kuldeep ◽  
S. Ashok ◽  
M. Satheesh
Keyword(s):  
Drug Delivery ◽  
Solid Lipid Nanoparticles ◽  
Large Scale ◽  
Colloidal Particles ◽  
Scale Up ◽  
Lipid Nanoparticles ◽  
Scale Production ◽  
Research Activity ◽  
New Approach ◽  
Solid Lipid

One of the situations in the treatment of disease is the delivery of efficacious medication of appropriate concentration to the site of action in a controlled and continual manner. Nanoparticle represents an important particulate carrier system, developed accordingly. Nanoparticles are solid colloidal particles ranging in size from 1 to 1000 nm and composed of macromolecular material. Nanoparticles could be polymeric or lipidic (SLNs). Industry estimates suggest that approximately 40% of lipophilic drug candidates fail due to solubility and formulation stability issues, prompting significant research activity in advanced lipophile delivery technologies. Solid lipid nanoparticle technology represents a promising new approach to lipophile drug delivery. Solid lipid nanoparticles (SLNs) are important advancement in this area. The bioacceptable and biodegradable nature of SLNs makes them less toxic as compared to polymeric nanoparticles. Supplemented with small size which prolongs the circulation time in blood, feasible scale up for large scale production and absence of burst effect makes them interesting candidates for study. In this present review this new approach is discussed in terms of their preparation, advantages, characterization and special features.

Sign in / Sign up

Export Citation Format

Share Document