Mode Pursuing Sampling Method for Discrete Variable Optimization on Expensive Black-Box Functions

2008 ◽  
Vol 130 (2) ◽  
Author(s):  
Behnam Sharif ◽  
G. Gary Wang ◽  
Tarek Y. ElMekkawy
Keyword(s):  
Global Optimization ◽  
Optimization Problems ◽  
Continuous Variable ◽  
Black Box ◽  
Test Problems ◽  
Continuous Variables ◽  
Optimization Strategy ◽  
Discrete Variable ◽  
Mps Method ◽  
Search Control

Based on previously developed Mode Pursuing Sampling (MPS) approach for continuous variables, a variation of MPS for discrete variable global optimization problems on expensive black-box functions is developed in this paper. The proposed method, namely, the discrete variable MPS (D-MPS) method, differs from its continuous variable version not only on sampling in a discrete space, but moreover, on a novel double-sphere strategy. The double-sphere strategy features two hyperspheres whose radii are dynamically enlarged or shrunk in control of, respectively, the degree of “exploration” and “exploitation” in the search of the optimum. Through testing and application to design problems, the proposed D-MPS method demonstrates excellent efficiency and accuracy as compared to the best results in literature on the test problems. The proposed method is believed a promising global optimization strategy for expensive black-box functions with discrete variables. The double-sphere strategy provides an original search control mechanism and has potential to be used in other search algorithms.

Trust Region Based Mode Pursuing Sampling Method for Global Optimization of High Dimensional Design Problems

2015 ◽  
Vol 137 (2) ◽  
Author(s):  
George H. Cheng ◽  
Adel Younis ◽  
Kambiz Haji Hajikolaei ◽  
G. Gary Wang
Keyword(s):  
Global Optimization ◽  
Optimization Problems ◽  
Trust Region ◽  
Standard Test ◽  
Black Box ◽  
High Dimensional ◽  
Efficient Global Optimization ◽  
Test Problems ◽  
Design Problems ◽  
Design Variables

Mode pursuing sampling (MPS) was developed as a global optimization algorithm for design optimization problems involving expensive black box functions. MPS has been found to be effective and efficient for design problems of low dimensionality, i.e., the number of design variables is less than 10. This work integrates the concept of trust regions into the MPS framework to create a new algorithm, trust region based mode pursuing sampling (TRMPS2), with the aim of dramatically improving performance and efficiency for high dimensional problems. TRMPS2 is benchmarked against genetic algorithm (GA), dividing rectangles (DIRECT), efficient global optimization (EGO), and MPS using a suite of standard test problems and an engineering design problem. The results show that TRMPS2 performs better on average than GA, DIRECT, EGO, and MPS for high dimensional, expensive, and black box (HEB) problems.

Improved Trust Region Based MPS Method for High-Dimensional Expensive Black-Box Problems

2013 ◽  
Author(s):  
George H. Cheng ◽  
Adel Younis ◽  
Kambiz Haji Hajikolaei ◽  
G. Gary Wang
Keyword(s):  
Optimization Problems ◽  
Trust Region ◽  
Black Box ◽  
High Dimensional ◽  
Test Problems ◽  
Design Variables ◽  
Low Dimensionality ◽  
Improve Algorithm ◽  
Improved Performance ◽  
Better Than

Mode Pursuing Sampling (MPS) was developed as a global optimization algorithm for optimization problems involving expensive black box functions. MPS has been found to be effective and efficient for problems of low dimensionality, i.e., the number of design variables is less than ten. A previous conference publication integrated the concept of trust regions into the MPS framework to create a new algorithm, TRMPS, which dramatically improved performance and efficiency for high dimensional problems. However, although TRMPS performed better than MPS, it was unproven against other established algorithms such as GA. This paper introduces an improved algorithm, TRMPS2, which incorporates guided sampling and low function value criterion to further improve algorithm performance for high dimensional problems. TRMPS2 is benchmarked against MPS and GA using a suite of test problems. The results show that TRMPS2 performs better than MPS and GA on average for high dimensional, expensive, and black box (HEB) problems.

Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints

2019 ◽  
Vol 31 (4) ◽  
pp. 689-702 ◽  
Author(s):  
Juliane Müller ◽  
Marcus Day
Keyword(s):  
Objective Function ◽  
Optimization Problems ◽  
Black Box ◽  
Direct Search ◽  
Test Problems ◽  
Large Set ◽  
Mesh Adaptive Direct Search ◽  
Surrogate Optimization ◽  
Computationally Expensive ◽  
Hidden Constraints

We introduce the algorithm SHEBO (surrogate optimization of problems with hidden constraints and expensive black-box objectives), an efficient optimization algorithm that employs surrogate models to solve computationally expensive black-box simulation optimization problems that have hidden constraints. Hidden constraints are encountered when the objective function evaluation does not return a value for a parameter vector. These constraints are often encountered in optimization problems in which the objective function is computed by a black-box simulation code. SHEBO uses a combination of local and global search strategies together with an evaluability prediction function and a dynamically adjusted evaluability threshold to iteratively select new sample points. We compare the performance of our algorithm with that of the mesh-based algorithms mesh adaptive direct search (MADS, NOMAD [nonlinear optimization by mesh adaptive direct search] implementation) and implicit filtering and SNOBFIT (stable noisy optimization by branch and fit), which assigns artificial function values to points that violate the hidden constraints. Our numerical experiments for a large set of test problems with 2–30 dimensions and a 31-dimensional real-world application problem arising in combustion simulation show that SHEBO is an efficient solver that outperforms the other methods for many test problems.

A new Kriging–Bat Algorithm for solving computationally expensive black-box global optimization problems

2018 ◽  
Vol 51 (2) ◽  
pp. 265-285 ◽  
Author(s):  
Abdulbaset Saad ◽  
Zuomin Dong ◽  
Brad Buckham ◽  
Curran Crawford ◽  
Adel Younis ◽  
...  
Keyword(s):  
Global Optimization ◽  
Optimization Problems ◽  
Bat Algorithm ◽  
Black Box ◽  
Computationally Expensive

Discretizing Continuous Problems for Faster Global Convergence

2003 ◽  
Author(s):  
Madara Ogot ◽  
Sherif Aly
Keyword(s):  
Global Optimization ◽  
Mechanical Design ◽  
Test Problems ◽  
Additional Parameter ◽  
Continuous Variables ◽  
Discrete State ◽  
Continuous Space ◽  
Design Variables ◽  
Asme Journal ◽  
Efficient Exploration

Global optimization of mechanical design problems using heuristic methods such as Simulated annealing (SA) and genetic algorithms (GAs) have been able to find global or near-global minima where prior methods have failed. The use of these nongradient based methods allow the broad efficient exploration of multimodal design spaces that could be continuous, discrete or mixed. From a survey of articles in the ASME Journal of Mechanical Design over the last 10 years, we have observed that researchers will typically run these algorithms in continuous mode for problems that contain continuous design variables. What we suggest in this paper is that computational efficiencies can be significantly increased by discretizing all continuous variables, perform a global optimization on the discretized design space, and then conduct a local search in the continuous space from the global minimum discrete state. The level of discretization will depend on the complexity of the problem, and becomes an additional parameter that needs to be tuned. The rational behind this assertion is presented, along with results from four test problems.

A New Global Optimization Method for Simultaneous Computation on Expensive Black-Box Functions

2003 ◽  
Author(s):  
Liqun Wang ◽  
Songqing Shan ◽  
G. Gary Wang
Keyword(s):  
Global Optimization ◽  
Optimization Problems ◽  
Optimization Methods ◽  
Search Space ◽  
Optimization Method ◽  
Global Optimum ◽  
Black Box ◽  
Efficient Global Optimization ◽  
Global Optimization Method ◽  
Global Optimization Methods

The presence of black-box functions in engineering design, which are usually computation-intensive, demands efficient global optimization methods. This work proposes a new global optimization method for black-box functions. The global optimization method is based on a novel mode-pursuing sampling (MPS) method which systematically generates more sample points in the neighborhood of the function mode while statistically covers the entire search space. Quadratic regression is performed to detect the region containing the global optimum. The sampling and detection process iterates until the global optimum is obtained. Through intensive testing, this method is found to be effective, efficient, robust, and applicable to both continuous and discontinuous functions. It supports simultaneous computation and applies to both unconstrained and constrained optimization problems. Because it does not call any existing global optimization tool, it can be used as a standalone global optimization method for inexpensive problems as well. Limitation of the method is also identified and discussed.

scholarly journals GOPS: efficient RBF surrogate global optimization algorithm with high dimensions and many parallel processors including application to multimodal water quality PDE model calibration

2020 ◽  
Author(s):  
Wei Xia ◽  
Christine Shoemaker
Keyword(s):  
Water Quality ◽  
Global Optimization ◽  
Objective Function ◽  
Black Box ◽  
High Dimensional ◽  
Function Evaluation ◽  
Test Problems ◽  
Quality Model ◽  
Lake Water Quality ◽  
Dimensional Parameter

Abstract This paper describes a new parallel global surrogate-based algorithm Global Optimization in Parallel with Surrogate (GOPS) for the minimization of continuous black-box objective functions that might have multiple local minima, are expensive to compute, and have no derivative information available. The task of picking P new evaluation points for P processors in each iteration is addressed by sampling around multiple center points at which the objective function has been previously evaluated. The GOPS algorithm improves on earlier algorithms by (a) new center points are selected based on bivariate non-dominated sorting of previously evaluated points with additional constraints to ensure the objective value is below a target percentile and (b) as iterations increase, the number of centers decreases, and the number of evaluation points per center increases. These strategies and the hyperparameters controlling them significantly improve GOPS’s parallel performance on high dimensional problems in comparison to other global optimization algorithms, especially with a larger number of processors. GOPS is tested with up to 128 processors in parallel on 14 synthetic black-box optimization benchmarking test problems (in 10, 21, and 40 dimensions) and one 21-dimensional parameter estimation problem for an expensive real-world nonlinear lake water quality model with partial differential equations that takes 22 min for each objective function evaluation. GOPS numerically significantly outperforms (especially on high dimensional problems and with larger numbers of processors) the earlier algorithms SOP and PSD-MADS-VNS (and these two algorithms have outperformed other algorithms in prior publications).

A Boolean Local Improvement Method for General Discrete Optimization Problems

1993 ◽  
Vol 115 (4) ◽  
pp. 776-783 ◽  
Author(s):  
Peng Lu ◽  
J. N. Siddall
Keyword(s):  
Optimization Problem ◽  
Optimization Problems ◽  
Continuous Variable ◽  
Local Region ◽  
Boolean Logic ◽  
Discrete Variable ◽  
New Approach ◽  
Local Improvement ◽  
Improvement Method ◽  
Discrete Optimization Problems

The paper proposes a new approach to solving the discrete design optimization problem by combining Boolean logic methods with conventional nonlinear optimization. A continuous variable optimum is first found, and then the Boolean method searches the local region for the discrete variable optimum.

AN IMPLEMENTATION OF A PARALLEL GENERALIZED BRANCH AND BOUND TEMPLATE

2007 ◽  
Vol 12 (3) ◽  
pp. 277-289 ◽  
Author(s):  
Milda Baravykaitė ◽  
Raimondas Čiegis
Keyword(s):  
Global Optimization ◽  
Load Balancing ◽  
Branch And Bound ◽  
Optimization Problems ◽  
Test Problems ◽  
Search Spaces ◽  
Parallel Version ◽  
Derivative Free ◽  
Template Library ◽  
Selection Of

Branch and bound (BnB) is a general algorithm to solve optimization problems. We present a template implementation of the BnB paradigm. A BnB template is implemented using C++ object oriented paradigm. MPI is used for underlying communications. A paradigm of domain decomposition (data parallelization) is used to construct a parallel algorithm. To obtain a better load balancing, the BnB template has the load balancing module that allows the redistribution of search spaces among the processors at run time. A parallel version of user's algorithm is obtained automatically. A new derivative-free global optimization algorithm is proposed for solving nonlinear global optimization problems. It is based on the BnB algorithm and its implementation is done by using the developed BnB algorithm template library. The robustness of the new algorithm is demonstrated by solving a selection of test problems.

Sign in / Sign up

Export Citation Format

Share Document