scholarly journals Developmental Bayesian Optimization of Black-Box with Visual Similarity-Based Transfer Learning

2018 ◽  
Author(s):  
Maxime Petit ◽  
Amaury Depierre ◽  
Xiaofang Wang ◽  
Emmanuel Dellandrea ◽  
Liming Chen
Keyword(s):  
Transfer Learning ◽  
Black Box ◽  
Visual Similarity ◽  
Bayesian Optimization

scholarly journals Multi-fidelity black-box optimization for time-optimal quadrotor maneuvers

2021 ◽  
pp. 027836492110333
Author(s):  
Gilhyun Ryou ◽  
Ezra Tal ◽  
Sertac Karaman
Keyword(s):  
Real World ◽  
High Speed ◽  
Analytical Approximation ◽  
Black Box ◽  
Bayesian Optimization ◽  
Optimization Framework ◽  
Quadrotor Aircraft ◽  
Time Optimal ◽  
Feasibility Constraints ◽  
Generation Problem

We consider the problem of generating a time-optimal quadrotor trajectory for highly maneuverable vehicles, such as quadrotor aircraft. The problem is challenging because the optimal trajectory is located on the boundary of the set of dynamically feasible trajectories. This boundary is hard to model as it involves limitations of the entire system, including complex aerodynamic and electromechanical phenomena, in agile high-speed flight. In this work, we propose a multi-fidelity Bayesian optimization framework that models the feasibility constraints based on analytical approximation, numerical simulation, and real-world flight experiments. By combining evaluations at different fidelities, trajectory time is optimized while the number of costly flight experiments is kept to a minimum. The algorithm is thoroughly evaluated for the trajectory generation problem in two different scenarios: (1) connecting predetermined waypoints; (2) planning in obstacle-rich environments. For each scenario, we conduct both simulation and real-world flight experiments at speeds up to 11 m/s. Resulting trajectories were found to be significantly faster than those obtained through minimum-snap trajectory planning.

Hyperparameter Optimization for Transfer Learning of VGG16 for Disease Identification in Corn Leaves Using Bayesian Optimization

Big Data ◽  
2021 ◽  
Author(s):  
Malliga Subramanian ◽  
Narasimha Prasad L.V. ◽  
Janakiramaiah B. ◽  
Mohan Babu A. ◽  
Sathishkumar V.E.
Keyword(s):  
Transfer Learning ◽  
Bayesian Optimization ◽  
Hyperparameter Optimization ◽  
Disease Identification

scholarly journals Transfer Learning based Impedance Identification of Voltage Source Converters

2021 ◽  
Author(s):  
Mengfan Zhang ◽  
xiongfei wang ◽  
Qianwen Xu
Keyword(s):  
Transfer Learning ◽  
Internal Control ◽  
Industrial Applications ◽  
Black Box ◽  
System Stability ◽  
Voltage Source ◽  
Voltage Source Converters ◽  
Impedance Model ◽  
Comparison Results ◽  
Stability Method

The black-box impedance of the voltage source converters (VSCs) can be directly identified at the converter terminal without access to its internal control details, which greatly facilitates the converter-grid interactions. However, since the limited impedance data amount in practical industrial applications, the existing impedance identification methods cannot accurately capture characteristics of the impedance model at various operating scenarios, which is the indicators of the VSCs system stability at the changing profiles of renewables and loads. In this paper, a transfer learning based impedance identification is proposed to fill this research gap. This method can significantly reduce the required data amount used in impedance identification so that the black-box impedance-based stability method could be applied for the practical industrial application. The comparison results confirm the accuracy of the impedance model obtained by this transfer learning based impedance identification method.

scholarly journals Black-box combinatorial optimization using models with integer-valued minima

2020 ◽  
Author(s):  
Laurens Bliek ◽  
Sicco Verwer ◽  
Mathijs de Weerdt
Keyword(s):  
Optimization Algorithm ◽  
Surrogate Model ◽  
Optimization Problems ◽  
Random Search ◽  
Continuous Optimization ◽  
Optimal Solution ◽  
Black Box ◽  
Combinatorial Problems ◽  
Bayesian Optimization ◽  
Bayesian Optimization Algorithm

Abstract When a black-box optimization objective can only be evaluated with costly or noisy measurements, most standard optimization algorithms are unsuited to find the optimal solution. Specialized algorithms that deal with exactly this situation make use of surrogate models. These models are usually continuous and smooth, which is beneficial for continuous optimization problems, but not necessarily for combinatorial problems. However, by choosing the basis functions of the surrogate model in a certain way, we show that it can be guaranteed that the optimal solution of the surrogate model is integer. This approach outperforms random search, simulated annealing and a Bayesian optimization algorithm on the problem of finding robust routes for a noise-perturbed traveling salesman benchmark problem, with similar performance as another Bayesian optimization algorithm, and outperforms all compared algorithms on a convex binary optimization problem with a large number of variables.

scholarly journals Modelling human active search in optimizing black-box functions

2020 ◽  
Vol 24 (23) ◽  
pp. 17771-17785
Author(s):  
Antonio Candelieri ◽  
Riccardo Perego ◽  
Ilaria Giordani ◽  
Andrea Ponti ◽  
Francesco Archetti
Keyword(s):  
Active Learning ◽  
Bayesian Learning ◽  
Surrogate Models ◽  
Black Box ◽  
Bayesian Optimization ◽  
Learning Approaches ◽  
Function Learning ◽  
The Subject ◽  
Human Function ◽  
Exploration Exploitation

AbstractModelling human function learning has been the subject of intense research in cognitive sciences. The topic is relevant in black-box optimization where information about the objective and/or constraints is not available and must be learned through function evaluations. In this paper, we focus on the relation between the behaviour of humans searching for the maximum and the probabilistic model used in Bayesian optimization. As surrogate models of the unknown function, both Gaussian processes and random forest have been considered: the Bayesian learning paradigm is central in the development of active learning approaches balancing exploration/exploitation in uncertain conditions towards effective generalization in large decision spaces. In this paper, we analyse experimentally how Bayesian optimization compares to humans searching for the maximum of an unknown 2D function. A set of controlled experiments with 60 subjects, using both surrogate models, confirm that Bayesian optimization provides a general model to represent individual patterns of active learning in humans.

scholarly journals Multi-Task Gaussian Process Upper Confidence Bound for Hyperparameter Tuning

2021 ◽  
Author(s):  
Bo Shen ◽  
Raghav Gnanasambandam ◽  
Rongxuan Wang ◽  
Zhenyu Kong
Keyword(s):  
Gaussian Process ◽  
Black Box ◽  
Bayesian Optimization ◽  
Query Point ◽  
Confidence Bound ◽  
Bayesian Optimization Algorithm ◽  
Machine Learning Model ◽  
Step Algorithm ◽  
Upper Confidence Bound ◽  
General Method

In many scientific and engineering applications, Bayesian optimization (BO) is a powerful tool for hyperparameter tuning of a machine learning model, materials design and discovery, etc. BO guides the choice of experiments in a sequential way to find a good combination of design points in as few experiments as possible. It can be formulated as a problem of optimizing a “black-box” function. Different from single-task Bayesian optimization, Multi-task Bayesian optimization is a general method to efficiently optimize multiple different but correlated “black-box” functions. The previous works in Multi-task Bayesian optimization algorithm queries a point to be evaluated for all tasks in each round of search, which is not efficient. For the case where different tasks are correlated, it is not necessary to evaluate all tasks for a given query point. Therefore, the objective of this work is to develop an algorithm for multi-task Bayesian optimization with automatic task selection so that only one task evaluation is needed per query round. Specifically, a new algorithm, namely, multi-task Gaussian process upper confidence bound (MT-GPUCB), is proposed to achieve this objective. The MT-GPUCB is a two-step algorithm, where the first step chooses which query point to evaluate, and the second step automatically selects the most informative task to evaluate. Under the bandit setting, a theoretical analysis is provided to show that our proposed MT-GPUCB is no-regret under some mild conditions. Our proposed algorithm is verified experimentally on a range of synthetic functions as well as real-world problems. The results clearly show the advantages of our query strategy for both design point and task.

scholarly journals Bayesian optimization with partially specified queries

2022 ◽  
Author(s):  
Shogo Hayashi ◽  
Junya Honda ◽  
Hisashi Kashima
Keyword(s):  
Black Box ◽  
Bayesian Optimization ◽  
Unknown Input ◽  
Bandit Problem ◽  
Test Functions ◽  
Significant Cost ◽  
Posterior Sampling ◽  
Regret Bounds ◽  
Input Variables ◽  
Real World Datasets

AbstractBayesian optimization (BO) is an approach to optimizing an expensive-to-evaluate black-box function and sequentially determines the values of input variables to evaluate the function. However, it is expensive and in some cases becomes difficult to specify values for all input variables, for example, in outsourcing scenarios where production of input queries with many input variables involves significant cost. In this paper, we propose a novel Gaussian process bandit problem, BO with partially specified queries (BOPSQ). In BOPSQ, unlike the standard BO setting, a learner specifies only the values of some input variables, and the values of the unspecified input variables are randomly determined according to a known or unknown distribution. We propose two algorithms based on posterior sampling for cases of known and unknown input distributions. We further derive their regret bounds that are sublinear for popular kernels. We demonstrate the effectiveness of the proposed algorithms using test functions and real-world datasets.

Decomposition of Black-Box Optimization Problems by Community Detection in Bayesian Networks

2012 ◽  
Vol 3 (4) ◽  
pp. 1-19 ◽  
Author(s):  
Marcio K. Crocomo ◽  
Jean P. Martins ◽  
Alexandre C. B. Delbem
Keyword(s):  
Probabilistic Models ◽  
Optimization Problems ◽  
Black Box ◽  
Point Of View ◽  
Bayesian Optimization ◽  
Genetic Operators ◽  
Step Method ◽  
Bayesian Optimization Algorithm ◽  
Probabilistic Sampling ◽  
Estimation Of Distribution

Estimation of Distribution Algorithms (EDAs) have proved themselves as an efficient alternative to Genetic Algorithms when solving nearly decomposable optimization problems. In general, EDAs substitute genetic operators by probabilistic sampling, enabling a better use of the information provided by the population and, consequently, a more efficient search. In this paper the authors exploit EDAs' probabilistic models from a different point-of-view, the authors argue that by looking for substructures in the probabilistic models it is possible to decompose a black-box optimization problem and solve it in a more straightforward way. Relying on the Building-Block hypothesis and the nearly-decomposability concept, their decompositional approach is implemented by a two-step method: 1) the current population is modeled by a Bayesian network, which is further decomposed into substructures (communities) using a version of the Fast Newman Algorithm. 2) Since the identified communities can be seen as sub-problems, they are solved separately and used to compose a solution for the original problem. The experiments showed strengths and limitations for the proposed method, but for some of the tested scenarios the authors’ method outperformed the Bayesian Optimization Algorithm by requiring up to 78% fewer fitness evaluations and being 30 times faster.

Sign in / Sign up

Export Citation Format

Share Document