Nonlinear switched capacitor 'neural' networks for optimization problems

1990 ◽  
Vol 37 (3) ◽  
pp. 384-398 ◽  
Author(s):  
A. Rodriguez-Vazquez ◽  
R. Dominguez-Castro ◽  
A. Rueda ◽  
J.L. Huertas ◽  
E. Sanchez-Sinencio
Keyword(s):  
Neural Networks ◽  
Optimization Problems ◽  
Switched Capacitor

Switched-capacitor neural networks using pulse based arithmetic

1990 ◽  
Vol 26 (15) ◽  
pp. 1118 ◽  
Author(s):  
B. Maundy ◽  
E. El-Masry
Keyword(s):  
Neural Networks ◽  
Switched Capacitor

Optimizing the Learning Process of Feedforward Neural Networks Using Lightning Search Algorithm

2016 ◽  
Vol 25 (06) ◽  
pp. 1650033 ◽  
Author(s):  
Hossam Faris ◽  
Ibrahim Aljarah ◽  
Nailah Al-Madi ◽  
Seyedali Mirjalili
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Optimization Technique ◽  
Back Propagation ◽  
Feedforward Neural Networks ◽  
Training Algorithms ◽  
Local Optima ◽  
Local Solutions

Evolutionary Neural Networks are proven to be beneficial in solving challenging datasets mainly due to the high local optima avoidance. Stochastic operators in such techniques reduce the probability of stagnation in local solutions and assist them to supersede conventional training algorithms such as Back Propagation (BP) and Levenberg-Marquardt (LM). According to the No-Free-Lunch (NFL), however, there is no optimization technique for solving all optimization problems. This means that a Neural Network trained by a new algorithm has the potential to solve a new set of problems or outperform the current techniques in solving existing problems. This motivates our attempts to investigate the efficiency of the recently proposed Evolutionary Algorithm called Lightning Search Algorithm (LSA) in training Neural Network for the first time in the literature. The LSA-based trainer is benchmarked on 16 popular medical diagnosis problems and compared to BP, LM, and 6 other evolutionary trainers. The quantitative and qualitative results show that the LSA algorithm is able to show not only better local solutions avoidance but also faster convergence speed compared to the other algorithms employed. In addition, the statistical test conducted proves that the LSA-based trainer is significantly superior in comparison with the current algorithms on the majority of datasets.

scholarly journals A Diffusion Approximation Theory of Momentum Stochastic Gradient Descent in Nonconvex Optimization

2021 ◽  
Author(s):  
Tianyi Liu ◽  
Zhehui Chen ◽  
Enlu Zhou ◽  
Tuo Zhao
Keyword(s):  
Neural Networks ◽  
Nonconvex Optimization ◽  
Gradient Descent ◽  
Deep Neural Networks ◽  
Optimization Problems ◽  
Saddle Points ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Nonconvex Optimization Problems ◽  
Empirical Success

Momentum stochastic gradient descent (MSGD) algorithm has been widely applied to many nonconvex optimization problems in machine learning (e.g., training deep neural networks, variational Bayesian inference, etc.). Despite its empirical success, there is still a lack of theoretical understanding of convergence properties of MSGD. To fill this gap, we propose to analyze the algorithmic behavior of MSGD by diffusion approximations for nonconvex optimization problems with strict saddle points and isolated local optima. Our study shows that the momentum helps escape from saddle points but hurts the convergence within the neighborhood of optima (if without the step size annealing or momentum annealing). Our theoretical discovery partially corroborates the empirical success of MSGD in training deep neural networks.

Convolutional Neural Networks for the Localization of Plastic Velocity Gradient Tensor in Polycrystalline Microstructures

2021 ◽  
pp. 1-41
Author(s):  
David Montes de Oca Zapiain ◽  
Apaar Shanker ◽  
Surya Kalidindi
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Velocity Gradient ◽  
Optimization Problems ◽  
Computational Cost ◽  
Two Phase ◽  
Design And Optimization ◽  
Gradient Fields ◽  
Generalized Spherical Harmonics ◽  
New Research

Abstract Recent work has demonstrated the potential of convolutional neural networks (CNNs) in producing low-computational cost surrogate models for the localization of mechanical fields in two-phase microstructures. The extension of the same CNNs to polycrystalline microstructures is hindered by the lack of an efficient formalism for the representation of the crystal lattice orientation in the input channels of the CNNs. In this paper, we demonstrate the benefits of using generalized spherical harmonics (GSH) for addressing this challenge. A CNN model was successfully trained to predict the local plastic velocity gradient fields in polycrystalline microstructures subjected to a macroscopically imposed loading condition. Specifically, it is demonstrated that the proposed approach improves significantly the accuracy of the CNN models, when compared with the direct use of Bunge-Euler angles to represent the crystal orientations in the input channels. Since the proposed approach implicitly satisfies the expected crystal symmetries in the specification of the input microstructure to the CNN, it opens new research directions for the adoption of CNNs in addressing a broad range of polycrystalline microstructure design and optimization problems.

MREM, Discrete Recurrent Network for Optimization

2011 ◽  
pp. 1112-1120
Author(s):  
Enrique Mérida-Casermeiro ◽  
Domingo López-Rodríguez ◽  
Juan M. Ortiz-de-Lazcano-Lobato
Keyword(s):  
Neural Networks ◽  
Combinatorial Optimization ◽  
Image Recognition ◽  
Single Neuron ◽  
Optimization Problems ◽  
Neural Model ◽  
Recurrent Network ◽  
Neural Models ◽  
Seminal Work ◽  
Combinatorial Optimization Problems

Since McCulloch and Pitts’ seminal work (McCulloch & Pitts, 1943), several models of discrete neural networks have been proposed, many of them presenting the ability of assigning a discrete value (other than unipolar or bipolar) to the output of a single neuron. These models have focused on a wide variety of applications. One of the most important models was developed by J. Hopfield in (Hopfield, 1982), which has been successfully applied in fields such as pattern and image recognition and reconstruction (Sun et al., 1995), design of analogdigital circuits (Tank & Hopfield, 1986), and, above all, in combinatorial optimization (Hopfield & Tank, 1985) (Takefuji, 1992) (Takefuji & Wang, 1996), among others. The purpose of this work is to review some applications of multivalued neural models to combinatorial optimization problems, focusing specifically on the neural model MREM, since it includes many of the multivalued models in the specialized literature.

Sign in / Sign up

Export Citation Format

Share Document