Neural networks as an aid to iterative optimization methods

2002 ◽  
Author(s):  
H.J. Li ◽  
A.H. Sung ◽  
W.W. Weiss ◽  
S.C. Wo
Keyword(s):  
Neural Networks ◽  
Optimization Methods ◽  
Iterative Optimization

scholarly journals Inverse Design for Silicon Photonics: From Iterative Optimization Algorithms to Deep Neural Networks

2021 ◽  
Vol 11 (9) ◽  
pp. 3822
Author(s):  
Simei Mao ◽  
Lirong Cheng ◽  
Caiyue Zhao ◽  
Faisal Nadeem Khan ◽  
Qian Li ◽  
...  
Keyword(s):  
Neural Networks ◽  
Silicon Photonics ◽  
Design Methodology ◽  
Degrees Of Freedom ◽  
Deep Neural Networks ◽  
Low Cost ◽  
Optimization Algorithms ◽  
Optimization Methods ◽  
Inverse Design ◽  
Iterative Optimization

Silicon photonics is a low-cost and versatile platform for various applications. For design of silicon photonic devices, the light-material interaction within its complex subwavelength geometry is difficult to investigate analytically and therefore numerical simulations are majorly adopted. To make the design process more time-efficient and to improve the device performance to its physical limits, various methods have been proposed over the past few years to manipulate the geometries of silicon platform for specific applications. In this review paper, we summarize the design methodologies for silicon photonics including iterative optimization algorithms and deep neural networks. In case of iterative optimization methods, we discuss them in different scenarios in the sequence of increased degrees of freedom: empirical structure, QR-code like structure and irregular structure. We also review inverse design approaches assisted by deep neural networks, which generate multiple devices with similar structure much faster than iterative optimization methods and are thus suitable in situations where piles of optical components are needed. Finally, the applications of inverse design methodology in optical neural networks are also discussed. This review intends to provide the readers with the suggestion for the most suitable design methodology for a specific scenario.

Optimization methods for deep neural networks

2021 ◽  
Author(s):  
Rajendra P. ◽  
Hanumantha Ravi. P. V. N. ◽  
Gunavardhana Naidu T.
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Optimization Methods

Application of various optimization methods in calculating the error of a neural network for diagnosing parasitological diseases of the gastrointestinal tract

2021 ◽  
Author(s):  
N.T. Abdullaev ◽  
U.N. Musevi ◽  
K.S. Pashaeva
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Gastrointestinal Tract ◽  
Optimization Methods ◽  
Optimization Method ◽  
Medical Diagnostics ◽  
The Body ◽  
Conjugate Gradients ◽  
Practical Significance ◽  
Parasitic Diseases

Formulation of the problem. This work is devoted to the use of artificial neural networks for diagnosing the functional state of the gastrointestinal tract caused by the influence of parasites in the body. For the experiment, 24 symptoms were selected, the number of which can be increased, and 9 most common diseases. The coincidence of neural network diagnostics with classical medical diagnostics for a specific disease is shown. The purpose of the work is to compare the neural networks in terms of their performance after describing the methods of preprocessing, isolating symptoms and classifying parasitic diseases of the gastrointestinal tract. Computer implementation of the experiment was carried out in the NeuroPro 0.25 software environment and optimization methods were chosen for training the network: "gradient descent" modified by Par Tan, "conjugate gradients", BFGS. Results. The results of forecasting using a multilayer perceptron using the above optimization methods are presented. To compare optimization methods, we used the values of the minimum and maximum network errors. Comparison of optimization methods using network errors makes it possible to draw the correct conclusion that for the task at hand, the best results were obtained when using the "conjugate gradients" optimization method. Practical significance. The proposed approach facilitates the work of the experimenter-doctor in choosing the optimization method when working with neural networks for the problem of diagnosing parasitic diseases of the gastrointestinal tract from the point of view of assessing the network error.

Comparison of Sensitivity Coefficient Calculation Methods in Automatic History Matching (includes associated paper 10873 )

1981 ◽  
Vol 21 (05) ◽  
pp. 551-557 ◽  
Author(s):  
Ali H. Dogru ◽  
John H. Seinfeld
Keyword(s):  
History Matching ◽  
Direct Method ◽  
Optimization Methods ◽  
Sensitivity Coefficient ◽  
Additional Time ◽  
Automatic History Matching ◽  
Matching Algorithm ◽  
Iterative Optimization ◽  
Incremental Change ◽  
Two Factors

Abstract The efficiency of automatic history matchingalgorithms depends on two factors: the computationtime needed per iteration and the number of iterations needed for convergence. In most historymatching algorithms, the most time-consumingaspect is the calculation of the sensitivitycoefficientsthe derivatives of the reservoir variables(pressure and saturation) with respect to the reservoirproperties (permeabilities and porosity). This paper presents an analysis of two methodsthe direct andthe variationalfor calculating sensitivitycoefficients, with particular emphasis on thecomputational requirements of the methods.If the simulator consists of a set of N ordinary differential equations for the grid-block variables(e.g., pressures)and there are M parameters forwhich the sensitivity coefficients are desired, the ratioof the computational efforts of the direct to thevariational method is N(M + 1)R = .N(N + 1) + M Thus, for M less than N the direct method is moreeconomical, whereas as M increases, a point isreached at which the variational method is preferred. Introduction There has been considerable interest in thedevelopment of automatic history matching algorithms.Although automatic history matching can offer significant advantages over trial-and-errorapproaches, its adoption has been somewhatlower than might have been anticipated when thefirst significant papers on the subject appeared. Oneobvious reason for the persistence of thetrial-and-error approach is that it does not requireadditional code development beyond that already involvedin the basic simulator, whereas automatic routinesrequire the appendixing of an iterative optimization routine to the basic simulator. Nevertheless, theinvestment of additional time in code developmentfor the history matching algorithm may be returned many fold during the actual history matchingexercise. In spite of the inherent advantages ofautomatic history matching, however, the automatic adjustment of the number of reservoir parameterstypically unknown even in a moderately sizedsimulation can require excessive amounts ofcomputation time. Therefore, it is of utmost importancethat an automatic history matching algorithm be asefficient as possible. Setting aside for the moment the issue of code complexity, the efficiency of analgorithm depends on two factors, the computationtime needed per iteration and the number ofiterations needed for convergence (whereconvergence is usually defined in terms of reaching acertain level of incremental change in either theparameters themselves or the objective function). Formost iterative optimization methods, the speed ofconvergence increases with the complexity of thealgorithm. SPEJ P. 551^

scholarly journals Meta-Heuristic Optimization Methods for Quaternion-Valued Neural Networks

Mathematics ◽  
2021 ◽  
Vol 9 (9) ◽  
pp. 938
Author(s):  
Jeremiah Bill ◽  
Lance Champagne ◽  
Bruce Cox ◽  
Trevor Bihl
Keyword(s):  
Neural Networks ◽  
Optimization Methods ◽  
Network Size ◽  
High Dimensional ◽  
Proof Of Concept ◽  
Activation Functions ◽  
Hypercomplex Numbers ◽  
Neural Network Structure ◽  
Future Work ◽  
High Dimensional Datasets

In recent years, real-valued neural networks have demonstrated promising, and often striking, results across a broad range of domains. This has driven a surge of applications utilizing high-dimensional datasets. While many techniques exist to alleviate issues of high-dimensionality, they all induce a cost in terms of network size or computational runtime. This work examines the use of quaternions, a form of hypercomplex numbers, in neural networks. The constructed networks demonstrate the ability of quaternions to encode high-dimensional data in an efficient neural network structure, showing that hypercomplex neural networks reduce the number of total trainable parameters compared to their real-valued equivalents. Finally, this work introduces a novel training algorithm using a meta-heuristic approach that bypasses the need for analytic quaternion loss or activation functions. This algorithm allows for a broader range of activation functions over current quaternion networks and presents a proof-of-concept for future work.

scholarly journals Thermal-based early breast cancer detection using inception V3, inception V4 and modified inception MV4

2021 ◽  
Author(s):  
Mohammed Abdulla Salim Al Husaini ◽  
Mohamed Hadi Habaebi ◽  
Teddy Surya Gunawan ◽  
Md Rafiqul Islam ◽  
Elfatih A. A. Elsheikh ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Early Stage ◽  
Computational Cost ◽  
Optimization Methods ◽  
Optimization Method ◽  
Deep Convolutional Neural Networks ◽  
Breast Thermography

AbstractBreast cancer is one of the most significant causes of death for women around the world. Breast thermography supported by deep convolutional neural networks is expected to contribute significantly to early detection and facilitate treatment at an early stage. The goal of this study is to investigate the behavior of different recent deep learning methods for identifying breast disorders. To evaluate our proposal, we built classifiers based on deep convolutional neural networks modelling inception V3, inception V4, and a modified version of the latter called inception MV4. MV4 was introduced to maintain the computational cost across all layers by making the resultant number of features and the number of pixel positions equal. DMR database was used for these deep learning models in classifying thermal images of healthy and sick patients. A set of epochs 3–30 were used in conjunction with learning rates 1 × 10–3, 1 × 10–4 and 1 × 10–5, Minibatch 10 and different optimization methods. The training results showed that inception V4 and MV4 with color images, a learning rate of 1 × 10–4, and SGDM optimization method, reached very high accuracy, verified through several experimental repetitions. With grayscale images, inception V3 outperforms V4 and MV4 by a considerable accuracy margin, for any optimization methods. In fact, the inception V3 (grayscale) performance is almost comparable to inception V4 and MV4 (color) performance but only after 20–30 epochs. inception MV4 achieved 7% faster classification response time compared to V4. The use of MV4 model is found to contribute to saving energy consumed and fluidity in arithmetic operations for the graphic processor. The results also indicate that increasing the number of layers may not necessarily be useful in improving the performance.

scholarly journals Elements of an algorithm for optimizing a parameter-structural neural network

2016 ◽  
Vol 101 (1) ◽  
pp. 27-35 ◽  
Author(s):  
Maria Mrówczyńska
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Optimization Methods ◽  
Group Method ◽  
Data Handling ◽  
Training Process ◽  
Measurement Results ◽  
Processing Information

Abstract The field of processing information provided by measurement results is one of the most important components of geodetic technologies. The dynamic development of this field improves classic algorithms for numerical calculations in the aspect of analytical solutions that are difficult to achieve. Algorithms based on artificial intelligence in the form of artificial neural networks, including the topology of connections between neurons have become an important instrument connected to the problem of processing and modelling processes. This concept results from the integration of neural networks and parameter optimization methods and makes it possible to avoid the necessity to arbitrarily define the structure of a network. This kind of extension of the training process is exemplified by the algorithm called the Group Method of Data Handling (GMDH), which belongs to the class of evolutionary algorithms. The article presents a GMDH type network, used for modelling deformations of the geometrical axis of a steel chimney during its operation.

Sign in / Sign up

Export Citation Format

Share Document