Development of a Conservative Model Validation Approach for Reliable Analysis

2015 ◽  
Author(s):  
Min-Yeong Moon ◽  
K. K. Choi ◽  
Hyunkyoo Cho ◽  
Nicholas Gaul ◽  
David Lamb ◽  
...  
Keyword(s):  
Experimental Data ◽  
Simulation Model ◽  
Model Validation ◽  
Simulation Models ◽  
Probability Of Failure ◽  
Simulation Based Design ◽  
Validation Measure ◽  
Conservative Estimation ◽  
The Cost ◽  
Target Output

Simulation models are approximations of real-world physical systems. Therefore, simulation model validation is necessary for the simulation-based design process to provide reliable products. However, due to the cost of product testing, experimental data in the context of model validation is limited for a given design. When the experimental data is limited, a true output PDF cannot be correctly obtained. Therefore, reliable target output PDF needs to be used to update the simulation model. In this paper, a new model validation approach is proposed to obtain a conservative estimation of the target output PDF for validation of the simulation model in reliability analysis. The proposed method considers the uncertainty induced by insufficient experimental data in estimation of predicted output PDFs by using Bayesian analysis. Then, a target output PDF and a probability of failure are selected from these predicted output PDFs at a user-specified conservativeness level for validation. For validation, the calibration parameter and model bias are optimized to minimize a validation measure of the simulation output PDF and the conservative target output PDF subject to the conservative probability of failure. For the optimization, accurate sensitivity of the validation measure is obtained using the complex variable method (CVM) for sensitivity analysis. As the target output PDF satisfies the user-specified conservativeness level, the validated simulation model provides a conservative representation of the experimental data. A simply supported beam is used to carry out the convergence study and demonstrate that the proposed method establishes a conservatively reliable simulation model.

Reliability-Based Design Optimization Using Confidence-Based Model Validation for Insufficient Experimental Data

2016 ◽  
Author(s):  
Min-Yeong Moon ◽  
K. K. Choi ◽  
Hyunkyoo Cho ◽  
Nicholas Gaul ◽  
David Lamb ◽  
...  
Keyword(s):  
Experimental Data ◽  
Simulation Model ◽  
Design Optimization ◽  
Model Validation ◽  
Model Bias ◽  
Reliability Based Design Optimization ◽  
Validation Process ◽  
Target Problem ◽  
Reliability Based Design ◽  
The Cost

The conventional reliability-based design optimization (RBDO) methods assume that a simulation model is able to represent the real physics accurately. However, the simulation model could be biased. Accordingly, when the conventional RBDO design is manufactured, the product may not satisfy the target reliability. Therefore, model validation, which corrects model bias, should be integrated in the RBDO process by incorporating experimental data. The challenge is that only a limited number of experimental data is usually available due to the cost of actual product testing. Consequently, model validation for RBDO needs to account for the uncertainty induced by insufficient experimental data as well as variability inherently existing in the products. In this paper, a confidence-based model validation process that captures the uncertainty and corrects model bias at user-specified target conservativeness level is developed. Thus, RBDO can be performed using confidence-based model validation to obtain conservative RBDO design. It is found that RBDO with model bias correction becomes a moving-target problem because the feasible domain changes as the design moves. Consequently, the RBDO optimum may not be easily found due to the convergence problem. To resolve the issue, an efficient process is proposed by carrying out deterministic design optimization (DDO) and RBDO without validation, followed by RBDO with confidence-based model validation. Finally, we demonstrate that the proposed RBDO approach can achieve a conservative and practical optimum design given a limited number of experimental data.

scholarly journals The General Principles of the Transportation Simulation Model Development and Validation

2020 ◽  
Vol 15 ◽  
Keyword(s):  
Simulation Model ◽  
Model Validation ◽  
Management Strategies ◽  
Model Development ◽  
Simulation Models ◽  
Model Verification ◽  
Microscopic Simulation ◽  
Calibration Methods ◽  
Transportation Simulation ◽  
Development And Validation

Transportation simulation model development allows simulating traveller’s decisions, evaluating various transportation management strategies and complex solutions. The aim of the paper is to set the general principles of the transportation simulation model development and validation. The paper contains the overview of the transportation simulation models types with the examples from the conducted projects for the Riga city. The basic steps of the simulation model development procedure: initial data preparation and analysis, transportation model development and simulation, scenarios planning and evaluation, and simulation models outcomes evaluation are considered. Simulation model verification, validation and calibration definitions are given. The basic checks for the transportation macroscopic and microscopic simulation model validation are listed. A summary of the transportation simulation model validation and calibration methods and parameters is given.

scholarly journals Some Issues in the Testing of Computer Simulation Models

2016 ◽  
Vol 5 (1) ◽  
pp. 1-10
Author(s):  
David Murray-Smith
Keyword(s):  
Simulation Model ◽  
Model Validation ◽  
Simulation Models ◽  
Model Testing ◽  
Model Verification ◽  
Verification And Validation ◽  
Quality Data ◽  
Model Quality ◽  
Face Validation

The testing of simulation models has much in common with testing processes in other types of application involving software development. However, there are also important differences associated with the fact that simulation model testing involves two distinct aspects, which are known as verification and validation. Model validation is concerned with investigation of modelling errors and model limitations while verification involves checking that the simulation program is an accurate representation of the mathematical and logical structure of the underlying model. Success in model validation depends upon the availability of detailed information about all aspects of the system being modelled. It also may depend on the availability of high quality data from the system which can be used to compare its behaviour with that of the corresponding simulation model. Transparency, high standards of documentation and good management of simulation models and data sets are basic requirements in simulation model testing. Unlike most other areas of software testing, model validation often has subjective elements, with potentially important contributions from face- validation procedures in which experts give a subjective assessment of the fidelity of the model. Verification and validation processes are not simply applied once but must be used repeatedly throughout the model development process, with regressive testing principles being applied. Decisions about when a model is acceptable for the intended application inevitably involve some form of risk assessment. A case study concerned with the development and application of a simulation model of a hydro-turbine and electrical generator system is used to illustrate some of the issues arising in a typical control engineering application. Results from the case study suggest that it is important to bring together objective aspects of simulation model testing and the more subjective face- validation aspects in a coherent fashion. Suggestions are also made about the need for changes in approach in the teaching of simulation techniques to engineering students to give more emphasis to issues of model quality, testing and validation.

Reliability-Based Design Optimization Using Confidence-Based Model Validation for Insufficient Experimental Data

2017 ◽  
Vol 139 (3) ◽  
Author(s):  
Min-Yeong Moon ◽  
K. K. Choi ◽  
Hyunkyoo Cho ◽  
Nicholas Gaul ◽  
David Lamb ◽  
...  
Keyword(s):  
Experimental Data ◽  
Simulation Model ◽  
Design Optimization ◽  
Model Validation ◽  
Optimum Design ◽  
Confidence Level ◽  
Optimization Procedure ◽  
Model Bias ◽  
Reliability Based Design Optimization ◽  
Reliability Based Design

The conventional reliability-based design optimization (RBDO) methods assume that a simulation model is able to represent the real physics accurately. However, this assumption may not always hold as the simulation model could be biased. Accordingly, designed product based on the conventional RBDO optimum may either not satisfy the target reliability or be overly conservative design. Therefore, simulation model validation using output experimental data, which corrects model bias, should be integrated in the RBDO process. With particular focus on RBDO, the model validation needs to account for the uncertainty induced by insufficient experimental data as well as the inherent variability of the products. In this paper, a confidence-based model validation method that captures the variability and the uncertainty, and that corrects model bias at a user-specified target confidence level, has been developed. The developed model validation helps RBDO to obtain a conservative RBDO optimum design at the target confidence level. The RBDO with model validation may have a convergence issue because the feasible domain changes as the design moves (i.e., a moving-target problem). To resolve the issue, a practical optimization procedure is proposed. Furthermore, the efficiency is achieved by carrying out deterministic design optimization (DDO) and RBDO without model validation, followed by RBDO with confidence-based model validation. Finally, we demonstrate that the proposed RBDO approach can achieve a conservative and practical optimum design given a limited number of experimental data.

scholarly journals Guidelines to Calibrate a Multi-Residential Building Simulation Model Addressing Overheating Evaluation and Residents’ Influence

Buildings ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 242
Author(s):  
Christoph Schünemann ◽  
David Schiela ◽  
Regine Ortlepp
Keyword(s):  
Simulation Model ◽  
Time Window ◽  
Residential Building ◽  
Simulation Models ◽  
Building Simulation ◽  
Night Time ◽  
Building Performance Simulation ◽  
Starting Point ◽  
3D Simulation Model ◽  
Building Physics

Can building performance simulation reproduce measured summertime indoor conditions of a multi-residential building in good conformity? This question is answered by calibrating simulated to monitored room temperatures of several rooms of a multi-residential building for an entire summer in two process steps. First, we did a calibration for several days without the residents being present to validate the building physics of the 3D simulation model. Second, the simulations were calibrated for the entire summer period, including the residents’ impact on evolving room temperature and overheating. As a result, a high degree of conformity between simulation and measurement could be achieved for all monitored rooms. The credibility of our results was secured by a detailed sensitivity analysis under varying meteorological conditions, shading situations, and window ventilation or room use in the simulation model. For top floor dwellings, a high overheating intensity was evoked by a combination of insufficient use of night-time window ventilation and non-heat-adapted residential behavior in combination with high solar gains and low heat storage capacities. Finally, the overall findings were merged into a process guideline to describe how a step-by-step calibration of residential building simulation models can be done. This guideline is intended to be a starting point for future discussions about the validity of the simplified boundary conditions which are often used in present-day standard overheating assessment.

Settings, Experimentation and Evaluation of the Simulation Models

2013 ◽  
Vol 309 ◽  
pp. 366-371 ◽  
Author(s):  
František Manlig ◽  
Radek Havlik ◽  
Alena Gottwaldova
Keyword(s):  
Simulation Model ◽  
New Technology ◽  
Simulation Models ◽  
Manufacturing Processes ◽  
Customer Requirements ◽  
Evaluation Functions ◽  
Operational Management ◽  
Minimum Number ◽  
User Friendly ◽  
Setting Parameters

This paper deals with research in computer simulation of manufacturing processes. The paper summarizes the procedures associated with developing the model, experimenting with and evaluating the model results. The key area is of experimentation with the simulation model and evaluation using indicators or multi-criteria functions. With regards to the experiment the crucial variables are the simulation model. The key ideas are to set the number of variables, depending on what a given simulation will be. For example, when introducing new technology into production, modify the type of warehouse, saving workers, thus economizing. The simulation models for the operational management uses simplified models, if possible, a minimum number of variables to obtain the result in shortest possible time. These models are more user friendly and the course will be conducted mostly in the background. An example of a criteria function is the number of parts produced or production time. Multi-criteria function has given us the opportunity to make better quality decisions. It is based on the composition of several parameters, including their weight to one end point. The type of evaluation functions, whether it is an indicator or criteria function is selected and based on customer requirements. In most cases it is recommended to use the multi-dimensional function. It gives us a more comprehensive view of the results from the model and facilitates decision-making. The result of this paper is a display of setting parameters for the experimentation on a sample model. Furthermore, the comparisons of results with a multi-criteria objective function and one-criterion indicator.

A Method to Define the Best Weld Sequence Using a Limited Number of Welding Simulation Analysis

2015 ◽  
Author(s):  
Mahyar Asadi ◽  
Ghazi Alsoruji
Keyword(s):  
Simulation Model ◽  
Surrogate Model ◽  
Simulation Analysis ◽  
Combinatorial Problem ◽  
Fem Analysis ◽  
Simulation Models ◽  
Computational Time ◽  
Optimal Sequence ◽  
Welded Structure ◽  
Welding Sequence

Weld sequence optimization, which is determining the best (and worst) welding sequence for welding work pieces, is a very common problem in welding design. The solution for such a combinatorial problem is limited by available resources. Although there are fast simulation models that support sequencing design, still it takes long because of many possible combinations, e.g. millions in a welded structure involving 10 passes. It is not feasible to choose the optimal sequence by evaluating all possible combinations, therefore this paper employs surrogate modeling that partially explores the design space and constructs an approximation model from some combinations of solutions of the expensive simulation model to mimic the behavior of the simulation model as closely as possible but at a much lower computational time and cost. This surrogate model, then, could be used to approximate the behavior of the other combinations and to find the best (and worst) sequence in terms of distortion. The technique is developed and tested on a simple panel structure with 4 weld passes, but essentially can be generalized to many weld passes. A comparison between the results of the surrogate model and the full transient FEM analysis all possible combinations shows the accuracy of the algorithm/model.

Rotorcraft Lateral-Directional Oscillations: The Anatomy of a Nuisance Mode

2021 ◽  
Author(s):  
Dheeraj Agarwal ◽  
Linghai Lu ◽  
Gareth D. Padfield ◽  
Mark D. White ◽  
Neil Cameron
Keyword(s):  
Simulation Model ◽  
Simulation Models ◽  
Flight Test ◽  
Flight Simulation ◽  
Stability And Control ◽  
Systems Research ◽  
Control Derivatives ◽  
Research Aircraft ◽  
Level Of Understanding ◽  
And Control

High-fidelity rotorcraft flight simulation relies on the availability of a quality flight model that further demands a good level of understanding of the complexities arising from aerodynamic couplings and interference effects. One such example is the difficulty in the prediction of the characteristics of the rotorcraft lateral-directional oscillation (LDO) mode in simulation. Achieving an acceptable level of the damping of this mode is a design challenge requiring simulation models with sufficient fidelity that reveal sources of destabilizing effects. This paper is focused on using System Identification to highlight such fidelity issues using Liverpool's FLIGHTLAB Bell 412 simulation model and in-flight LDO measurements from the bare airframe National Research Council's (Canada) Advanced Systems Research Aircraft. The simulation model was renovated to improve the fidelity of the model. The results show a close match between the identified models and flight test for the LDO mode frequency and damping. Comparison of identified stability and control derivatives with those predicted by the simulation model highlight areas of good and poor fidelity.

Sign in / Sign up

Export Citation Format

Share Document