Model Validation Metric and Model Bias Characterization for Dynamic System Responses Under Uncertainty

Quantification of the accuracy of analytical models (math or computer simulation models) and characterization of the model bias are two essential processes in model validation. Available model validation metrics, whether qualitative or quantitative, do not consider the influence of the number of experimental data for model accuracy check. In addition, quantitative measure from the validation metric does not directly reflect the level of model accuracy, i.e. from 0% to 100%, especially when there is a lack of experimental data. If the original model prediction does not satisfy accuracy criteria compared to the experimental data, instead of revising the model conceptually, characterization of the model bias may be a more practical approach to improve the model accuracy because there is probably no ideal model which can predict the actual physical system with no error. So far, there is a lack of effective approaches that can accurately characterize the model bias for multiple dynamic system responses. To overcome these limitations, the first objective of this study is to develop a model validation metric for model accuracy check considering different number of experimental data. Specifically, a validation metric using the Bhattacharya distance (B-distance) is proposed with three notable benefits. First of all, the metric directly compares the distributions of two set of uncertain system responses from model prediction and experiment rather than the distribution parameters (e.g. mean and variance). Second, the B-distance quantitatively measures the degree of accuracy from 0% to 100% between the distributions of the uncertain system responses. Third, reference accuracy metric with respect to different number of experimental data can be effectively obtained so that hypothesis test can be performed to identify whether the two distributions are identical or not in a probability manner. The second objective of this study is to propose an effective approach to accurately characterize the model bias for dynamic system responses. Specially, the model bias is represented by a generic random process, where realizations of the model bias at each time step could follow arbitrary distributions. Instead of using the traditional Bayesian or Maximum Likelihood Estimation (MLE) approach, we propose a novel and efficient approach to identify the model bias using a generic random process modeling technique. A vehicle safety system with 11 dynamic system responses is used to demonstrate the effectiveness of the proposed approach.