On the Taxonomy of Optimization Problems Under Estimation of Distribution Algorithms

2013 ◽  
Vol 21 (3) ◽  
pp. 471-495 ◽  
Author(s):  
Carlos Echegoyen ◽  
Alexander Mendiburu ◽  
Roberto Santana ◽  
Jose A. Lozano
Keyword(s):  
Probabilistic Model ◽  
Hamming Distance ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Search Space ◽  
Necessary Condition ◽  
Estimation Of Distribution Algorithms ◽  
Local Optima ◽  
Estimation Of Distribution ◽  
Distribution Algorithms

Understanding the relationship between a search algorithm and the space of problems is a fundamental issue in the optimization field. In this paper, we lay the foundations to elaborate taxonomies of problems under estimation of distribution algorithms (EDAs). By using an infinite population model and assuming that the selection operator is based on the rank of the solutions, we group optimization problems according to the behavior of the EDA. Throughout the definition of an equivalence relation between functions it is possible to partition the space of problems in equivalence classes in which the algorithm has the same behavior. We show that only the probabilistic model is able to generate different partitions of the set of possible problems and hence, it predetermines the number of different behaviors that the algorithm can exhibit. As a natural consequence of our definitions, all the objective functions are in the same equivalence class when the algorithm does not impose restrictions to the probabilistic model. The taxonomy of problems, which is also valid for finite populations, is studied in depth for a simple EDA that considers independence among the variables of the problem. We provide the sufficient and necessary condition to decide the equivalence between functions and then we develop the operators to describe and count the members of a class. In addition, we show the intrinsic relation between univariate EDAs and the neighborhood system induced by the Hamming distance by proving that all the functions in the same class have the same number of local optima and that they are in the same ranking positions. Finally, we carry out numerical simulations in order to analyze the different behaviors that the algorithm can exhibit for the functions defined over the search space [Formula: see text].

The Limits of Estimation of Distribution Algorithms

2014 ◽  
Vol 926-930 ◽  
pp. 3294-3297
Author(s):  
Cai Chang Ding ◽  
Wen Xiu Peng ◽  
Wei Ming Wang
Keyword(s):  
Bayesian Networks ◽  
Probabilistic Model ◽  
Estimation Of Distribution Algorithms ◽  
Case Scenario ◽  
Learning Methods ◽  
Worst Case ◽  
Worst Case Scenario ◽  
Estimation Of Distribution ◽  
Distribution Algorithms

In this paper, we study the ability limit of EDAs to effectively solve problems in relation to the number of interactions among the variables. More in particular, we numerically analyze the learning limits that different EDA implementations encounter to solve problems on a sequence of additively decomposable functions (ADFs) in which new sub-functions are progressively added. The study is carried out in a worst-case scenario where the sub-functions are defined as deceptive functions. We argue that the limits for this type of algorithm are mainly imposed by the probabilistic model they rely on. Beyond the limitations of the approximate learning methods, the results suggest that, in general, the use of bayesian networks can entail strong computational restrictions to overcome the limits of applicability.

scholarly journals Can Compact Optimisation Algorithms Be Structurally Biased?

2020 ◽  
Author(s):  
Anna V. Kononova ◽  
Fabio Caraffini ◽  
Hao Wang ◽  
Thomas Bäck
Keyword(s):  
Search Space ◽  
Test Cases ◽  
Estimation Of Distribution Algorithms ◽  
Test Function ◽  
Stochastic Optimisation ◽  
Estimation Of Distribution ◽  
Distribution Algorithms ◽  
Anderson Darling ◽  
Optimisation Algorithms

In the field of stochastic optimisation, the so-called structural bias constitutes an undesired behaviour of an algorithm that is unable to explore the search space to a uniform extent. In this paper, we investigate whether algorithms from a subclass of estimation of distribution algorithms, the compact algorithms, exhibit structural bias. Our approach, justified in our earlier publications, is based on conducting experiments on a test function whose values are uniformly distributed in its domain. For the experiment, 81 combinations of compact algorithms and strategies of dealing with infeasible solutions have been selected as test cases. We have applied two approaches for determining the presence and severity of structural bias, namely a visual and a statistical (Anderson-Darling) tests. Our results suggest that compact algorithms are more immune to structural bias than their counterparts maintaining explicit populations. Both tests indicate that strong structural bias is found only in one of the algorithms (cBFO) regardless of the choice of strategy of dealing with infeasible solutions and cPSO mirror. For other test cases, statistical and visual tests disagree on some cases classified as having mild or strong structural bias: the former one tends to make harsher decisions, thus needing further investigation.

Drift and Scaling in Estimation of Distribution Algorithms

2005 ◽  
Vol 13 (1) ◽  
pp. 99-123 ◽  
Author(s):  
J. L. Shapiro
Keyword(s):  
Population Size ◽  
Probability Model ◽  
Search Space ◽  
System Size ◽  
Estimation Of Distribution Algorithms ◽  
Square Root ◽  
Problem Size ◽  
Probability Models ◽  
Estimation Of Distribution ◽  
Distribution Algorithms

This paper considers a phenomenon in Estimation of Distribution Algorithms (EDA) analogous to drift in population genetic dynamics. Finite population sampling in selection results in fluctuations which get reinforced when the probability model is updated. As a consequence, any probability model which can generate only a single set of values with probability 1 can be an attractive fixed point of the algorithm. To avoid this, parameters of the algorithm must scale with the system size in strongly problem-dependent ways, or the algorithm must be modified. This phenomenon is shown to hold for general EDAs as a consequence of the lack of ergodicity and irreducibility of the Markov chain on the state of probability models. It is illustrated in the case of UMDA, in which it is shown that the global optimum is only found if the population size is sufficiently large. For the needle-in-a haystack problem, the population size must scale as the square-root of the size of the search space. For the one-max problem, the population size must scale as the square-root of the problem size.

scholarly journals Properties of Gray and Binary Representations

2004 ◽  
Vol 12 (1) ◽  
pp. 47-76 ◽  
Author(s):  
Jonathan Rowe ◽  
Darrell Whitley ◽  
Laura Barbulescu ◽  
Jean-Paul Watson
Keyword(s):  
Genetic Algorithms ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Search Space ◽  
Gray Code ◽  
Global Optimum ◽  
Gray Codes ◽  
Neighborhood Structure ◽  
Local Optima ◽  
Vertex Set

Representations are formalized as encodings that map the search space to the vertex set of a graph. We define the notion of bit equivalent encodings and show that for such encodings the corresponding Walsh coefficients are also conserved. We focus on Gray codes as particular types of encoding and present a review of properties related to the use of Gray codes. Gray codes are widely used in conjunction with genetic algorithms and bit-climbing algorithms for parameter optimization problems. We present new convergence proofs for a special class of unimodal functions; the proofs show that a steepest ascent bit climber using any reflected Gray code representation reaches the global optimum in a number of steps that is linear with respect to the encoding size. There are in fact many different Gray codes.Shifting is defined as a mechanism for dynamically switching from one Gray code representation to another in order to escape local optima. Theoretical results that substantially improve our understanding of the Gray codes and the shifting mechanism are presented. New proofs also shed light on the number of unique Gray code neighborhoods accessible via shifting and on how neighborhood structure changes during shifting. We show that shifting can improve the performance of both a local search algorithm as well as one of the best genetic algorithms currently available.

Estimation of Distribution Algorithms as Logistic Regression Regularizers of Microarray Classifiers

2009 ◽  
Vol 48 (03) ◽  
pp. 236-241 ◽  
Author(s):  
V. Robles ◽  
P. Larrañaga ◽  
C. Bielza
Keyword(s):  
Logistic Regression ◽  
Microarray Data ◽  
Optimization Problems ◽  
Likelihood Function ◽  
Recursive Feature Elimination ◽  
Parameter Estimates ◽  
Data Sets ◽  
Estimation Of Distribution Algorithms ◽  
Estimation Of Distribution ◽  
Distribution Algorithms

Summary Objectives: The “large k (genes), small N (samples)” phenomenon complicates the problem of microarray classification with logistic regression. The indeterminacy of the maximum likelihood solutions, multicollinearity of predictor variables and data over-fitting cause unstable parameter estimates. Moreover, computational problems arise due to the large number of predictor (genes) variables. Regularized logistic regression excels as a solution. However, the difficulties found here involve an objective function hard to be optimized from a mathematical viewpoint and a careful required tuning of the regularization parameters. Methods: Those difficulties are tackled by introducing a new way of regularizing the logistic regression. Estimation of distribution algorithms (EDAs), a kind of evolutionary algorithms, emerge as natural regularizers. Obtaining the regularized estimates of the logistic classifier amounts to maximizing the likelihood function via our EDA, without having to be penalized. Likelihood penalties add a number of difficulties to the resulting optimization problems, which vanish in our case. Simulation of new estimates during the evolutionary process of EDAs is performed in such a way that guarantees their shrinkage while maintaining their probabilistic dependence relationships learnt. The EDA process is embedded in an adapted recursive feature elimination procedure, thereby providing the genes that are best markers for the classification. Results: The consistency with the literature and excellent classification performance achieved with our algorithm are illustrated on four microarray data sets: Breast, Colon, Leukemia and Prostate. Details on the last two data sets are available as supplementary material. Conclusions: We have introduced a novel EDA-based logistic regression regularizer. It implicitly shrinks the coefficients during EDA evolution process while optimizing the usual likelihood function. The approach is combined with a gene subset selection procedure and automatically tunes the required parameters. Empirical results on microarray data sets provide sparse models with confirmed genes and performing better in classification than other competing regularized methods.

Globally Multimodal Problem Optimization Via an Estimation of Distribution Algorithm Based on Unsupervised Learning of Bayesian Networks

2005 ◽  
Vol 13 (1) ◽  
pp. 43-66 ◽  
Author(s):  
J. M. Peña ◽  
J. A. Lozano ◽  
P. Larrañaga
Keyword(s):  
Bayesian Networks ◽  
Unsupervised Learning ◽  
Genetic Drift ◽  
Optimization Problems ◽  
Estimation Of Distribution Algorithm ◽  
Estimation Of Distribution Algorithms ◽  
Estimation Of Distribution ◽  
Global Optima ◽  
Effectiveness And Efficiency ◽  
Distribution Algorithms

Many optimization problems are what can be called globally multimodal, i.e., they present several global optima. Unfortunately, this is a major source of difficulties for most estimation of distribution algorithms, making their effectiveness and efficiency degrade, due to genetic drift. With the aim of overcoming these drawbacks for discrete globally multimodal problem optimization, this paper introduces and evaluates a new estimation of distribution algorithm based on unsupervised learning of Bayesian networks. We report the satisfactory results of our experiments with symmetrical binary optimization problems.

Sign in / Sign up

Export Citation Format

Share Document