scholarly journals Flattening Layer Pruning in Convolutional Neural Networks

Symmetry ◽  
2021 ◽  
Vol 13 (7) ◽  
pp. 1147
Author(s):  
Ernest Jeczmionek ◽  
Piotr A. Kowalski
Keyword(s):  
Neural Networks ◽  
Network Topology ◽  
Global Sensitivity Analysis ◽  
Global Sensitivity ◽  
The Neural Network ◽  
Input Variables ◽  
The Impact ◽  
Different Levels ◽  
Pruning Methods

The rapid growth of performance in the field of neural networks has also increased their sizes. Pruning methods are getting more and more attention in order to overcome the problem of non-impactful parameters and overgrowth of neurons. In this article, the application of Global Sensitivity Analysis (GSA) methods demonstrates the impact of input variables on the model’s output variables. GSA gives the ability to mark out the least meaningful arguments and build reduction algorithms on these. Using several popular datasets, the study shows how different levels of pruning correlate to network accuracy and how levels of reduction negligibly impact accuracy. In doing so, pre- and post-reduction sizes of neural networks are compared. This paper shows how Sobol and FAST methods with common norms can largely decrease the size of a network, while keeping accuracy relatively high. On the basis of the obtained results, it is possible to create a thesis about the asymmetry between the elements removed from the network topology and the quality of the neural network.

Analytical Variance-Based Global Sensitivity Analysis in Simulation-Based Design Under Uncertainty

2004 ◽  
Author(s):  
Wei Chen ◽  
Ruichen Jin ◽  
Agus Sudjianto
Keyword(s):  
Sensitivity Analysis ◽  
Global Sensitivity Analysis ◽  
Simulation Models ◽  
Design Under Uncertainty ◽  
Global Sensitivity ◽  
Functional Relationships ◽  
Simulation Based Design ◽  
Simulation Based ◽  
Input Variables ◽  
The Impact

The importance of sensitivity analysis in engineering design cannot be over-emphasized. In design under uncertainty, sensitivity analysis is performed with respect to the probabilistic characteristics. Global sensitivity analysis (GSA), in particular, is used to study the impact of variations in input variables on the variation of a model output. One of the most challenging issues for GSA is the intensive computational demand for assessing the impact of probabilistic variations. Existing variance-based GSA methods are developed for general functional relationships but require a large number of samples. In this work, we develop an efficient and accurate approach to GSA that employs analytic formulations derived from metamodels of engineering simulation models. We examine the types of GSA needed for design under uncertainty and derive generalized analytical formulations of GSA based on a variety of metamodels commonly used in engineering applications. The benefits of our proposed techniques are demonstrated and verified through both illustrative mathematical examples and the robust design for improving vehicle handling performance.

scholarly journals Multi-expert annotation of Crohn’s disease images of the small bowel for automatic detection using a convolutional recurrent attention neural network

2021 ◽  
Vol 09 (07) ◽  
pp. E1136-E1144
Author(s):  
Astrid de Maissin ◽  
Remi Vallée ◽  
Mathurin Flamant ◽  
Marie Fondain-Bossiere ◽  
Catherine Le Berre ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Crohn’S Disease ◽  
Crohn's Disease ◽  
Diagnostic Tools ◽  
The Neural Network ◽  
Before And After ◽  
Expert Annotation ◽  
The Impact

Abstract Background and study aims Computer-aided diagnostic tools using deep neural networks are efficient for detection of lesions in endoscopy but require a huge number of images. The impact of the quality of annotation has not been tested yet. Here we describe a multi-expert annotated dataset of images extracted from capsules from Crohn’s disease patients and the impact of the quality of annotations on the accuracy of a recurrent attention neural network. Methods Images of capsule were annotated by a reader first and then reviewed by three experts in inflammatory bowel disease. Concordance analysis between experts was evaluated by Fleiss’ kappa and all the discordant images were, again, read by all the endoscopists to obtain a consensus annotation. A recurrent attention neural network developed for the study was tested before and after the consensus annotation. Available neural networks (ResNet and VGGNet) were also tested under the same conditions. Results The final dataset included 3498 images with 2124 non-pathological (60.7 %), 1360 pathological (38.9 %), and 14 (0.4 %) inconclusive. Agreement of the experts was good for distinguishing pathological and non-pathological images with a kappa of 0.79 (P < 0.0001). The accuracy of our classifier and the available neural networks increased after the consensus annotation with a precision of 93.7 %, sensitivity of 93 %, and specificity of 95 %. Conclusions The accuracy of the neural network increased with improved annotations, suggesting that the number of images needed for the development of these systems could be diminished using a well-designed dataset.

Application of 0D model of blood flow to study vessel abnormalities in the human systemic circulation: An in-silico study

2018 ◽  
Vol 11 (08) ◽  
pp. 1850106 ◽  
Author(s):  
R. Gul ◽  
A. Shahzad ◽  
M. Zubair
Keyword(s):  
Sensitivity Analysis ◽  
Blood Flow ◽  
Global Sensitivity Analysis ◽  
Carotid Bifurcation ◽  
Clinical Decision ◽  
Systemic Circulation ◽  
Wave Pattern ◽  
Global Sensitivity ◽  
The Impact ◽  
Different Levels

In this paper, a multi-compartment 0D model of the blood flow is considered to study the vessel abnormalities (stenoses and aneurysms) in the human systemic circulation (SC). In the complete SC, different levels of stenosis and aneurysms are artificially created by decreasing and increasing the vessel diameters respectively and their effects on pressure and flow are studied using sensitivity analysis (SA). Normalized local sensitivity analysis (LSA) is used to study the impact of stenosis and aneurysms on pressure and flow wave pattern. Furthermore, global sensitivity analysis (GSA), Sobol’s method is used to quantify the overall influence of stenoses and aneurysms in the complete SC. The results of global sensitivity analysis revealed that the impact of both stenoses and aneurysms is strong within the individual structures (arm, legs, carotid bifurcation, aorta), while, aortic stenoses and aneurysms have effect on almost all downstream nodes. Moreover, the study could be useful for medical doctors, teachers and students to observe the hemodynamical changes in the SC with respect to vessel abnormalities, which could further help in making any clinical decision for patients having different levels of vessel abnormalities in any part or structure of the SC.

Global Sensitivity Analysis for Field Response Based on the Manifold of Feature Covariance Matrix

2021 ◽  
Author(s):  
Zhouzhou Song ◽  
Zhao Liu ◽  
Can Xu ◽  
Ping Zhu
Keyword(s):  
Sensitivity Analysis ◽  
Covariance Matrix ◽  
Computational Models ◽  
Global Sensitivity Analysis ◽  
Feature Selection Method ◽  
Global Sensitivity ◽  
Sensitivity Indices ◽  
Field Response ◽  
Input Variables ◽  
The Impact

Abstract In real-world applications, it is commonplace that the computational models have field responses, i.e., the temporal or spatial fields. It has become a critical task to develop global sensitivity analysis (GSA) methods to measure the effect of each input variable on the full-field. In this paper, a new sensitivity analysis method based on the manifold of feature covariance matrix (FCM) is developed for quantifying the impact of input variables on the field response. The method firstly performs feature extraction on the field response to obtain a low-dimensional FCM. An adaptive feature selection method is proposed to avoid the FCM from singularity. Thereby, the field response is represented by a FCM, which lies on a symmetric positive-definite matrix manifold. Then, the GSA technique based on the Cramér-von Mises distance for output valued on the Riemannian manifold is introduced for estimating the sensitivity indices for field response. An example of a temporal field and an example of a 2-D displacement field are introduced to demonstrate the applicability of the proposed method in estimating global sensitivity indices for field solution. Results show that the proposed method can distinguish the important input variables correctly and can yield robust index values. Besides, the proposed method can be implemented for GSA for field responses of different dimensionalities.

scholarly journals A Novel Global Sensitivity Measure Based on Probability Weighted Moments

Symmetry ◽  
2021 ◽  
Vol 13 (1) ◽  
pp. 90
Author(s):  
Shufang Song ◽  
Lu Wang
Keyword(s):  
Sensitivity Analysis ◽  
Global Sensitivity Analysis ◽  
Small Samples ◽  
Double Loop ◽  
Numerical Estimation ◽  
Global Sensitivity ◽  
Sensitivity Measure ◽  
Probability Weighted Moments ◽  
Input Variables ◽  
Single Set

Global sensitivity analysis (GSA) is a useful tool to evaluate the influence of input variables in the whole distribution range. Variance-based methods and moment-independent methods are widely studied and popular GSA techniques despite their several shortcomings. Since probability weighted moments (PWMs) include more information than classical moments and can be accurately estimated from small samples, a novel global sensitivity measure based on PWMs is proposed. Then, two methods are introduced to estimate the proposed measure, i.e., double-loop-repeated-set numerical estimation and double-loop-single-set numerical estimation. Several numerical and engineering examples are used to show its advantages.

Nonintrusive Global Sensitivity Analysis for Linear Systems With Process Noise

2019 ◽  
Vol 14 (2) ◽  
Author(s):  
Souransu Nandi ◽  
Tarunraj Singh
Keyword(s):  
Sensitivity Analysis ◽  
Linear Systems ◽  
Global Sensitivity Analysis ◽  
Benchmark Problems ◽  
Model Parameters ◽  
Parameter Uncertainties ◽  
Sobol Indices ◽  
Global Sensitivity ◽  
Time Invariant ◽  
The Impact

The focus of this paper is on the global sensitivity analysis (GSA) of linear systems with time-invariant model parameter uncertainties and driven by stochastic inputs. The Sobol' indices of the evolving mean and variance estimates of states are used to assess the impact of the time-invariant uncertain model parameters and the statistics of the stochastic input on the uncertainty of the output. Numerical results on two benchmark problems help illustrate that it is conceivable that parameters, which are not so significant in contributing to the uncertainty of the mean, can be extremely significant in contributing to the uncertainty of the variances. The paper uses a polynomial chaos (PC) approach to synthesize a surrogate probabilistic model of the stochastic system after using Lagrange interpolation polynomials (LIPs) as PC bases. The Sobol' indices are then directly evaluated from the PC coefficients. Although this concept is not new, a novel interpretation of stochastic collocation-based PC and intrusive PC is presented where they are shown to represent identical probabilistic models when the system under consideration is linear. This result now permits treating linear models as black boxes to develop intrusive PC surrogates.

Fundamentals of Higher Order Neural Networks for Modeling and Simulation

2013 ◽  
pp. 103-133 ◽  
Author(s):  
Madan M. Gupta ◽  
Ivo Bukovsky ◽  
Noriyasu Homma ◽  
Ashu M. G. Solo ◽  
Zeng-Guang Hou
Keyword(s):  
Neural Networks ◽  
Modeling And Simulation ◽  
Learning Algorithm ◽  
Nonlinear Approximation ◽  
Higher Order ◽  
Higher Order Neural Networks ◽  
High Dynamic ◽  
Input Variables ◽  
Continuous Dynamic

In this chapter, the authors provide fundamental principles of Higher Order Neural Units (HONUs) and Higher Order Neural Networks (HONNs) for modeling and simulation. An essential core of HONNs can be found in higher order weighted combinations or correlations between the input variables and HONU. Except for the high quality of nonlinear approximation of static HONUs, the capability of dynamic HONUs for the modeling of dynamic systems is shown and compared to conventional recurrent neural networks when a practical learning algorithm is used. In addition, the potential of continuous dynamic HONUs to approximate high dynamic order systems is discussed, as adaptable time delays can be implemented. By using some typical examples, this chapter describes how and why higher order combinations or correlations can be effective for modeling of systems.

Efficient and Accurate Global Sensitivity Analysis for Reservoir Simulations By Use of the Probabilistic Collocation Method

SPE Journal ◽  
2013 ◽  
Vol 19 (04) ◽  
pp. 621-635 ◽  
Author(s):  
Cheng Dai ◽  
Heng Li ◽  
Dongxiao Zhang
Keyword(s):  
Sensitivity Analysis ◽  
Collocation Method ◽  
History Matching ◽  
Global Sensitivity Analysis ◽  
Computational Cost ◽  
Global Sensitivity ◽  
Reservoir Simulations ◽  
Probabilistic Collocation Method ◽  
The Impact ◽  
Probabilistic Collocation

Summary Reservoir simulations involve a large number of formation and fluid parameters, many of which are subject to uncertainties owing to the combination of spatial heterogeneity and insufficient measurements. Accurately quantifying the impact of varying parameters on simulation models can reveal the importance of the parameters, which helps in designing field-characterization strategies and determining parameterization for history matching. Compared with the commonly used local sensitivity analysis (SA), global SA considers the whole variation range of the parameters and can thus provide more-complete information. However, the traditional global sensitivity analysis that is derived from Monte Carlo simulation (MCS) is computationally too demanding for reservoir simulations. In this study, we propose an alternative approach that is both accurate and efficient. In the proposed approach, the model outputs such as pressure and reservoir production quantities are expressed by polynomial chaos expansions (PCEs). The probabilistic collocation method is used to determine the coefficients of the polynomial expansions by solving outputs at different sets of collocation points by means of the original partial-differential equations. Then, a proxy is constructed with such coefficients. Accurate statistical sensitivity indices of the uncertainty parameters can be obtained by running the proxy. We validate the approach with 2D examples by comparing with the MCS-based global SA. It is found that with only a small fraction of the computational cost required by the MCS approach, the new approach gives accurate global sensitivity for each parameter. The proposed approach is also demonstrated on a large-scale 3D black-oil model, for which the MCS-based global SA is found to be computationally infeasible. It is found that the developed approach possesses the following key advantages: It requires a much smaller number of reservoir simulations for accurate global SA; it is nonintrusive and can be implemented with existing codes or simulators; and it can accommodate arbitrary distributions of parameters encountered in realistic geological situations.

scholarly journals Global Sensitivity Analysis to Assess Salt Precipitation for CO2 Geological Storage in Deep Saline Aquifers

Geofluids ◽  
2017 ◽  
Vol 2017 ◽  
pp. 1-16 ◽  
Author(s):  
Yuan Wang ◽  
Jie Ren ◽  
Shaobin Hu ◽  
Di Feng
Keyword(s):  
Sensitivity Analysis ◽  
Global Sensitivity Analysis ◽  
Injection Rate ◽  
Injection Well ◽  
Saline Aquifers ◽  
Salt Precipitation ◽  
Empirical Parameter ◽  
Global Sensitivity ◽  
Morris Method ◽  
Input Variables

Salt precipitation is generated near the injection well when dry supercritical carbon dioxide (scCO2) is injected into saline aquifers, and it can seriously impair the CO2 injectivity of the well. We used solid saturation (Ss) to map CO2 injectivity. Ss was used as the response variable for the sensitivity analysis, and the input variables included the CO2 injection rate (QCO2), salinity of the aquifer (XNaCl), empirical parameter m, air entry pressure (P0), maximum capillary pressure (Pmax), and liquid residual saturation (Splr and Sclr). Global sensitivity analysis methods, namely, the Morris method and Sobol method, were used. A significant increase in Ss was observed near the injection well, and the results of the two methods were similar: XNaCl had the greatest effect on Ss; the effect of P0 and Pmax on Ss was negligible. On the other hand, with these two methods, QCO2 had various effects on Ss: QCO2 had a large effect on Ss in the Morris method, but it had little effect on Ss in the Sobol method. We also found that a low QCO2 had a profound effect on Ss but that a high QCO2 had almost no effect on the Ss value.

Sign in / Sign up

Export Citation Format

Share Document