Adaptive Orthonormal Basis Functions for High Dimensional Metamodeling With Existing Sample Points

2012 ◽  
Author(s):  
Kambiz Haji Hajikolaei ◽  
G. Gary Wang
Keyword(s):  
Orthonormal Basis ◽  
Black Box ◽  
Basis Functions ◽  
High Dimensional ◽  
High Dimensional Model Representation ◽  
Design Practice ◽  
Test Functions ◽  
Sample Data ◽  
Sample Points ◽  
Relative Errors

High Dimensional Model Representation (HDMR) is a tool for generating an approximation of an input-output model for a multivariate function. It can be used to model a black-box function for metamodel-based optimization. Recently the authors’ team has developed a radial basis function based HDMR (RBF-HDMR) model that can efficiently model a high dimensional black-box function and, moreover, to uncover inner variable structures of the black-box function. This approach, however, requests a complete new, although optimized, set of sample points, as dictated by the methodology, while in engineering design practice one often has many existing sample data. How to utilize the existing data to efficiently construct a HDMR model is the focus of this paper. We first identify the Random-Sampling HDMR (RS-HDMR), which uses orthonormal basis functions as HDMR component functions and existing sample points can be used to calculate the coefficients of the basis functions. One of the important issues related to the RS-HDMR is that in theory the basis functions are obtained based on the continuous integrations related to the orthonormality conditions. In practice, however, the integrations are approximated by Monte Carlo summation and thus the basis functions may not satisfy the orthonormality conditions. In this paper, we propose new and adaptive orthonormal basis functions with respect to a given set of sample points for RS-HDMR approximation. RS-HDMR models are built for different test functions using the standard and new adaptive basis functions for different number of sample points. The relative errors for both models are calculated and compared. The results show that the models that are built using the new basis functions are more accurate.

A Global High Dimensional Metamodeling Approach With the Ability of Using Non-Uniform Sampling

2014 ◽  
Author(s):  
Kambiz Haji Hajikolaei ◽  
G. Gary Wang
Keyword(s):  
Linear Combination ◽  
Principal Component ◽  
Critical Issue ◽  
Basis Functions ◽  
High Dimensional ◽  
Benchmark Problems ◽  
High Dimensional Model Representation ◽  
Uniform Sampling ◽  
Non Uniform Sampling ◽  
Sample Points

In engineering design, spending excessive amount of time on physical experiments or expensive simulations makes the design costly and lengthy. This issue exacerbates when the design problem has a large number of inputs, or of high dimension. High Dimensional Model Representation (HDMR) is one powerful method in approximating high dimensional, expensive, black-box (HEB) problems. One existing HDMR implementation, Random Sampling HDMR (RS-HDMR), can build a HDMR model from random sample points with a linear combination of basis functions. The most critical issue in RS-HDMR is that calculating the coefficients for the basis functions includes integrals that are approximated by Monte Carlo summations, which are error prone with limited samples and especially with non-uniform sampling. In this paper, a new approach based on Principal Component Analysis (PCA), called PCA-HDMR, is proposed for finding the coefficients that provide the best linear combination of the bases with minimum error and without using any integral. Benchmark problems are modeled using the method and the results are compared with RS-HDMR results. With both uniform and non-uniform sampling, PCA-HDMR built more accurate models than RS-HDMR for a given set of sample points.

Polynomial Representation of the Gaussian Process

2018 ◽  
Author(s):  
Jesper Kristensen ◽  
Isaac Asher ◽  
Liping Wang
Keyword(s):  
Gaussian Process ◽  
Basis Functions ◽  
Training Data ◽  
Computational Time ◽  
High Dimensional ◽  
High Dimensional Model Representation ◽  
Analysis Tool ◽  
Sobol Indices ◽  
Robust Fitting ◽  
Uncertainty Estimates

Gaussian Process (GP) regression is a well-established probabilistic meta-modeling and data analysis tool. The posterior distribution of the GP parameters can be estimated using, e.g., Markov Chain Monte Carlo (MCMC). The ability to make predictions is a key aspect of using such surrogate models. To make a GP prediction, the MCMC chain as well as the training data are required. For some applications, GP predictions can require too much computational time and/or memory, especially for many training data points. This motivates the present work to represent the GP in an equivalent polynomial (or other global functional) form called a portable GP. The portable GP inherits many benefits of the GP including feature ranking via Sobol indices, robust fitting to non-linear and high-dimensional data, accurate uncertainty estimates, etc. The framework expands the GP in a high-dimensional model representation (HDMR). After fitting each HDMR basis function with a polynomial, they are all added together to form the portable GP. A ranking of which basis functions to use in the fitting process is automatically provided via Sobol indices. The uncertainty from the fitting process can be propagated to the final GP polynomial estimate. In applications where speed and accuracy are paramount, spline fits to the basis functions give very good results. Finally, portable BHM provides an alternative set of assumptions with regards to extrapolation behavior which may be more appropriate than the assumptions inherent in GPs.

Metamodeling for High Dimensional Simulation-Based Design Problems

2010 ◽  
Vol 132 (5) ◽  
Author(s):  
Songqing Shan ◽  
G. Gary Wang
Keyword(s):  
Practical Method ◽  
Black Box ◽  
High Dimensional ◽  
High Dimensional Model Representation ◽  
Element Analysis ◽  
Design Problems ◽  
Function Expression ◽  
Input Variables ◽  
Dimensional Simulation ◽  
Low Dimensional

Computational tools such as finite element analysis and simulation are widely used in engineering, but they are mostly used for design analysis and validation. If these tools can be integrated for design optimization, it will undoubtedly enhance a manufacturer’s competitiveness. Such integration, however, faces three main challenges: (1) high computational expense of simulation, (2) the simulation process being a black-box function, and (3) design problems being high dimensional. In the past two decades, metamodeling has been intensively developed to deal with expensive black-box functions, and has achieved success for low dimensional design problems. But when high dimensionality is also present in design, which is often found in practice, there lacks of a practical method to deal with the so-called high dimensional, expensive, and black-box (HEB) problems. This paper proposes the first metamodel of its kind to tackle the HEB problem. This paper integrates the radial basis function with high dimensional model representation into a new model, RBF-HDMR. The developed RBF-HDMR model offers an explicit function expression, and can reveal (1) the contribution of each design variable, (2) inherent linearity/nonlinearity with respect to input variables, and (3) correlation relationships among input variables. An accompanying algorithm to construct the RBF-HDMR has also been developed. The model and the algorithm fundamentally change the exponentially growing computation cost to be polynomial. Testing and comparison confirm the efficiency and capability of RBF-HDMR for HEB problems.

Turning Black-Box Into White Functions

2010 ◽  
Author(s):  
Songqing Shan ◽  
G. Gary Wang
Keyword(s):  
Functional Form ◽  
Black Box ◽  
High Dimensional ◽  
High Dimensional Model Representation ◽  
Significant Variable ◽  
Dimensional Model ◽  
Adaptive Process ◽  
Order Form ◽  
Effectiveness And Efficiency ◽  
Test Result

Modeling of high dimensional expensive black-box (HEB) functions is challenging. A recently developed method, radial basis function-based high dimensional model representation (RBF-HDMR), has been found promising. This work extends RBF-HDMR to enhance its modeling capability beyond the current second order form and “uncover” black-box functions so that not only a more accurate metamodel is obtained, but also key information of the function can be gained and thus the black-box function can be turned “white.” The key information that can be gained includes 1) functional form, 2) (non)linearity with respect to each variable, 3) variable correlations. The resultant model can be used for applications such as sensitivity analysis, visualization, and optimization. The RBF-HDMR exploration is based on identifying the existence of certain variable correlations through derived theorems. The adaptive process of exploration and modeling reveals the black-box functions till all significant variable correlations are found. The black-box functional form is then represented by a structure matrix that can manifest all orders of correlated behavior of variables. The proposed approach is tested with theoretical and practical examples. The test result demonstrates the effectiveness and efficiency of the proposed approach.

Optimization on Metamodeling-Supported Iterative Decomposition

2015 ◽  
Vol 138 (2) ◽  
Author(s):  
Kambiz Haji Hajikolaei ◽  
George H. Cheng ◽  
G. Gary Wang
Keyword(s):  
Trust Region ◽  
Principal Component ◽  
Global Optimum ◽  
High Dimensional ◽  
High Dimensional Model Representation ◽  
Step Method ◽  
Direct Optimization ◽  
Decomposition Scheme ◽  
New Strategy ◽  
Sample Points

The recently developed metamodel-based decomposition strategy relies on quantifying the variable correlations of black-box functions so that high-dimensional problems are decomposed to smaller subproblems, before performing optimization. Such a two-step method may miss the global optimum due to its rigidity or requires extra expensive sample points for ensuring adequate decomposition. This work develops a strategy to iteratively decompose high-dimensional problems within the optimization process. The sample points used during the optimization are reused to build a metamodel called principal component analysis-high dimensional model representation (PCA-HDMR) for quantifying the intensities of variable correlations by sensitivity analysis. At every iteration, the predicted intensities of the correlations are updated based on all the evaluated points, and a new decomposition scheme is suggested by omitting the weak correlations. Optimization is performed on the iteratively updated subproblems from decomposition. The proposed strategy is applied for optimization of different benchmarks and engineering problems, and results are compared to direct optimization of the undecomposed problems using trust region mode pursuing sampling method (TRMPS), genetic algorithm (GA), cooperative coevolutionary algorithm with correlation-based adaptive variable partitioning (CCEA-AVP), and divide rectangles (DIRECT). The results show that except for the category of undecomposable problems with all or many strong (i.e., important) correlations, the proposed strategy effectively improves the accuracy of the optimization results. The advantages of the new strategy in comparison with the previous methods are also discussed.

High Dimensional Model Representation With Principal Component Analysis

2013 ◽  
Vol 136 (1) ◽  
Author(s):  
Kambiz Haji Hajikolaei ◽  
G. Gary Wang
Keyword(s):  
Principal Component Analysis ◽  
Linear Combination ◽  
Principal Component ◽  
Component Analysis ◽  
Nonuniform Sampling ◽  
Basis Functions ◽  
Model Representation ◽  
High Dimensional ◽  
High Dimensional Model Representation ◽  
Dimensional Model

In engineering design, spending excessive amount of time on physical experiments or expensive simulations makes the design costly and lengthy. This issue exacerbates when the design problem has a large number of inputs, or of high dimension. High dimensional model representation (HDMR) is one powerful method in approximating high dimensional, expensive, black-box (HEB) problems. One existing HDMR implementation, random sampling HDMR (RS-HDMR), can build an HDMR model from random sample points with a linear combination of basis functions. The most critical issue in RS-HDMR is that calculating the coefficients for the basis functions includes integrals that are approximated by Monte Carlo summations, which are error prone with limited samples and especially with nonuniform sampling. In this paper, a new approach based on principal component analysis (PCA), called PCA-HDMR, is proposed for finding the coefficients that provide the best linear combination of the bases with minimum error and without using any integral. Several benchmark problems of different dimensionalities and one engineering problem are modeled using the method and the results are compared with RS-HDMR results. In all problems with both uniform and nonuniform sampling, PCA-HDMR built more accurate models than RS-HDMR for a given set of sample points.

Turning Black-Box Functions Into White Functions

2011 ◽  
Vol 133 (3) ◽  
Author(s):  
Songqing Shan ◽  
G. Gary Wang
Keyword(s):  
Functional Form ◽  
Black Box ◽  
High Dimensional ◽  
High Dimensional Model Representation ◽  
Significant Variable ◽  
Test Results ◽  
Dimensional Model ◽  
Adaptive Process ◽  
Modeling Process ◽  
Effectiveness And Efficiency

A recently developed metamodel, radial basis function-based high-dimensional model representation (RBF-HDMR), shows promise as a metamodel for high-dimensional expensive black-box functions. This work extends the modeling capability of RBF-HDMR from the current second-order form to any higher order. More importantly, the modeling process “uncovers” black-box functions so that not only is a more accurate metamodel obtained, but also key information about the function can be gained and thus the black-box function can be turned “white.” The key information that can be gained includes: (1) functional form, (2) (non)linearity with respect to each variable, and (3) variable correlations. The black-box “uncovering” process is based on identifying the existence of certain variable correlations through two derived theorems. The adaptive process of exploration and modeling reveals the black-box functions until all significant variable correlations are found. The black-box functional form is then represented by a structure matrix that can manifest all orders of correlated behavior of the variables. The resultant metamodel and its revealed inner structure lend themselves well to applications such as sensitivity analysis, decomposition, visualization, and optimization. The proposed approach is tested with theoretical and practical examples. The test results demonstrate the effectiveness and efficiency of the proposed approach.

Development of Adaptive RBF-HDMR Model for Approximating High Dimensional Problems

2009 ◽  
Author(s):  
Songqing Shan ◽  
G. Gary Wang
Keyword(s):  
Curse Of Dimensionality ◽  
Black Box ◽  
Divide And Conquer ◽  
High Dimensional ◽  
High Dimensional Model Representation ◽  
Dimensional Model ◽  
Adaptive Modeling ◽  
Computationally Expensive ◽  
New Form ◽  
Modeling Strategy

Modeling or approximating high dimensional, computationally-expensive, black-box problems faces an exponentially increasing difficulty, the “curse-of-dimensionality”. This paper proposes a new form of high-dimensional model representation (HDMR) by integrating the radial basis function (RBF). The developed model, called RBF-HDMR, naturally explores and exploits the linearity/nonlinearity and correlation relationships among variables of the underlying function that is unknown or computationally expensive. This work also derives a lemma that supports the divide-and-conquer and adaptive modeling strategy of RBF-HDMR. RBF-HDMR circumvents or alleviates the “curse-of-dimensionality” by means of its explicit hierarchical structure, adaptive modeling strategy tailored to inherent variable relation, sample reuse, and a divide-and-conquer space-filling sampling algorithm. Multiple mathematical examples of a wide scope of dimensionalities are given to illustrate the modeling principle, procedure, efficiency, and accuracy of RBF-HDMR.

Sign in / Sign up

Export Citation Format

Share Document