Theoretical Convergence Guarantees for Cooperative Coevolutionary Algorithms

2010 ◽  
Vol 18 (4) ◽  
pp. 581-615 ◽  
Author(s):  
Liviu Panait
Keyword(s):  
Theoretical Foundation ◽  
Formal Model ◽  
Optimal Solution ◽  
Theoretical Work ◽  
Search Space ◽  
Simple Problem ◽  
Search Process ◽  
Coevolutionary Algorithms ◽  
Suboptimal Solutions ◽  
Speed Up

Cooperative coevolutionary algorithms have the potential to significantly speed up the search process by dividing the space into parts that can each be conquered separately. However, recent research presented theoretical and empirical arguments that these algorithms tend to converge to suboptimal solutions in the search space, and are thus not fit for optimization tasks. This paper details an extended formal model for cooperative coevolutionary algorithms, and uses it to explore possible reasons these algorithms converge to optimal or suboptimal solutions. We demonstrate that, under specific conditions, this theoretical model will converge to the globally optimal solution. The proofs provide the underlying theoretical foundation for a better application of cooperative coevolutionary algorithms. We demonstrate the practical advantages of applying ideas from this theoretical work to a simple problem domain.

scholarly journals Evolutionary Squeaky Wheel Optimization: A New Framework for Analysis

2011 ◽  
Vol 19 (3) ◽  
pp. 405-428 ◽  
Author(s):  
Jingpeng Li ◽  
Andrew J. Parkes ◽  
Edmund K. Burke
Keyword(s):  
Simulated Annealing ◽  
Search Space ◽  
Simple Problem ◽  
Problem Instance ◽  
Search Process ◽  
Distribution Probability ◽  
Formal Framework ◽  
Recent Extension ◽  
New Framework ◽  
Real World Problems

Squeaky wheel optimization (SWO) is a relatively new metaheuristic that has been shown to be effective for many real-world problems. At each iteration SWO does a complete construction of a solution starting from the empty assignment. Although the construction uses information from previous iterations, the complete rebuilding does mean that SWO is generally effective at diversification but can suffer from a relatively weak intensification. Evolutionary SWO (ESWO) is a recent extension to SWO that is designed to improve the intensification by keeping the good components of solutions and only using SWO to reconstruct other poorer components of the solution. In such algorithms a standard challenge is to understand how the various parameters affect the search process. In order to support the future study of such issues, we propose a formal framework for the analysis of ESWO. The framework is based on Markov chains, and the main novelty arises because ESWO moves through the space of partial assignments. This makes it significantly different from the analyses used in local search (such as simulated annealing) which only move through complete assignments. Generally, the exact details of ESWO will depend on various heuristics; so we focus our approach on a case of ESWO that we call ESWO-II and that has probabilistic as opposed to heuristic selection and construction operators. For ESWO-II, we study a simple problem instance and explicitly compute the stationary distribution probability over the states of the search space. We find interesting properties of the distribution. In particular, we find that the probabilities of states generally, but not always, increase with their fitness. This nonmonotonocity is quite different from the monotonicity expected in algorithms such as simulated annealing.

One-Shot Neural Architecture Search by Dynamically Pruning Supernet in Hierarchical Order

2021 ◽  
pp. 2150029
Author(s):  
Jianwei Zhang ◽  
Dong Li ◽  
Lituan Wang ◽  
Lei Zhang
Keyword(s):  
State Of The Art ◽  
Search Space ◽  
Experimental Results ◽  
Research Interest ◽  
Search Process ◽  
Search Efficiency ◽  
Neural Architecture ◽  
Speed Up ◽  
The One

Neural Architecture Search (NAS), which aims at automatically designing neural architectures, recently draw a growing research interest. Different from conventional NAS methods, in which a large number of neural architectures need to be trained for evaluation, the one-shot NAS methods only have to train one supernet which synthesizes all the possible candidate architectures. As a result, the search efficiency could be significantly improved by sharing the supernet’s weights during the candidate architectures’ evaluation. This strategy could greatly speed up the search process but suffer a challenge that the evaluation based on sharing weights is not predictive enough. Recently, pruning the supernet during the search has been proven to be an efficient way to alleviate this problem. However, the pruning direction in complex-structured search space remains unexplored. In this paper, we revisited the role of path dropout strategy, which drops the neural operations instead of the neurons, in supernet training, and several interesting characters of the supernet trained with dropout are found. Based on the observations, a Hierarchically-Ordered Pruning Neural Architecture Search (HOPNAS) algorithm is proposed by dynamically pruning the supernet with a proper pruning direction. Experimental results indicate that our method is competitive with state-of-the-art approaches on CIFAR10 and ImageNet.

Whale optimization algorithm based on lateral inhibition for image matching and vision-guided AUV docking

2020 ◽  
pp. 1-12
Author(s):  
Zheping Yan ◽  
Jinzhong Zhang ◽  
Jialing Tang
Keyword(s):  
Lateral Inhibition ◽  
Optimization Algorithm ◽  
Image Matching ◽  
Autonomous Underwater Vehicle ◽  
Optimal Solution ◽  
Search Space ◽  
Whale Optimization Algorithm ◽  
Inhibition Mechanism ◽  
Whale Optimization ◽  
Accuracy And Stability

The accuracy and stability of relative pose estimation of an autonomous underwater vehicle (AUV) and a target depend on whether the characteristics of the underwater image can be accurately and quickly extracted. In this paper, a whale optimization algorithm (WOA) based on lateral inhibition (LI) is proposed to solve the image matching and vision-guided AUV docking problem. The proposed method is named the LI-WOA. The WOA is motivated by the behavior of humpback whales, and it mainly imitates encircling prey, bubble-net attacking and searching for prey to obtain the globally optimal solution in the search space. The WOA not only balances exploration and exploitation but also has a faster convergence speed, higher calculation accuracy and stronger robustness than other approaches. The lateral inhibition mechanism can effectively perform image enhancement and image edge extraction to improve the accuracy and stability of image matching. The LI-WOA combines the optimization efficiency of the WOA and the matching accuracy of the LI mechanism to improve convergence accuracy and the correct matching rate. To verify its effectiveness and feasibility, the WOA is compared with other algorithms by maximizing the similarity between the original image and the template image. The experimental results show that the LI-WOA has a better average value, a higher correct rate, less execution time and stronger robustness than other algorithms. The LI-WOA is an effective and stable method for solving the image matching and vision-guided AUV docking problem.

scholarly journals Quit When You Can: Efficient Evaluation of Ensembles by Optimized Ordering

2021 ◽  
Vol 17 (4) ◽  
pp. 1-20
Author(s):  
Serena Wang ◽  
Maya Gupta ◽  
Seungil You
Keyword(s):  
Optimization Problem ◽  
Optimal Solution ◽  
Combinatorial Optimization Problem ◽  
Joint Optimization ◽  
Early Stopping ◽  
Time Approximation ◽  
Approximation Bound ◽  
Evaluation Time ◽  
Speed Up ◽  
Real World Problems

Given a classifier ensemble and a dataset, many examples may be confidently and accurately classified after only a subset of the base models in the ensemble is evaluated. Dynamically deciding to classify early can reduce both mean latency and CPU without harming the accuracy of the original ensemble. To achieve such gains, we propose jointly optimizing the evaluation order of the base models and early-stopping thresholds. Our proposed objective is a combinatorial optimization problem, but we provide a greedy algorithm that achieves a 4-approximation of the optimal solution under certain assumptions, which is also the best achievable polynomial-time approximation bound. Experiments on benchmark and real-world problems show that the proposed Quit When You Can (QWYC) algorithm can speed up average evaluation time by 1.8–2.7 times on even jointly trained ensembles, which are more difficult to speed up than independently or sequentially trained ensembles. QWYC’s joint optimization of ordering and thresholds also performed better in experiments than previous fixed orderings, including gradient boosted trees’ ordering.

scholarly journals Optimum Distribution System Expansion Planning Incorporating DG Based on N-1 Criterion for Sustainable System

2021 ◽  
Vol 13 (12) ◽  
pp. 6708
Author(s):  
Hamza Mubarak ◽  
Nurulafiqah Nadzirah Mansor ◽  
Hazlie Mokhlis ◽  
Mahazani Mohamad ◽  
Hasmaini Mohamad ◽  
...  
Keyword(s):  
Power Systems ◽  
Distribution System ◽  
Industrial Revolution ◽  
Optimal Solution ◽  
Search Space ◽  
Planning Problem ◽  
System Expansion ◽  
Optimum Distribution ◽  
Optimal Sizing ◽  
Expansion Planning

Demand for continuous and reliable power supply has significantly increased, especially in this Industrial Revolution 4.0 era. In this regard, adequate planning of electrical power systems considering persistent load growth, increased integration of distributed generators (DGs), optimal system operation during N-1 contingencies, and compliance to the existing system constraints are paramount. However, these issues need to be parallelly addressed for optimum distribution system planning. Consequently, the planning optimization problem would become more complex due to the various technical and operational constraints as well as the enormous search space. To address these considerations, this paper proposes a strategy to obtain one optimal solution for the distribution system expansion planning by considering N-1 system contingencies for all branches and DG optimal sizing and placement as well as fluctuations in the load profiles. In this work, a hybrid firefly algorithm and particle swarm optimization (FA-PSO) was proposed to determine the optimal solution for the expansion planning problem. The validity of the proposed method was tested on IEEE 33- and 69-bus systems. The results show that incorporating DGs with optimal sizing and location minimizes the investment and power loss cost for the 33-bus system by 42.18% and 14.63%, respectively, and for the 69-system by 31.53% and 12%, respectively. In addition, comparative studies were done with a different model from the literature to verify the robustness of the proposed method.

scholarly journals Temporal concatenation for Markov decision processes

2021 ◽  
pp. 1-28
Author(s):  
Ruiyang Song ◽  
Kuang Xu
Keyword(s):  
Markov Decision Processes ◽  
Large Scale ◽  
Optimal Solution ◽  
Upper Bounds ◽  
Black Box ◽  
Decision Processes ◽  
Optimal Solutions ◽  
Wide Range ◽  
Markov Decision ◽  
Speed Up

We propose and analyze a temporal concatenation heuristic for solving large-scale finite-horizon Markov decision processes (MDP), which divides the MDP into smaller sub-problems along the time horizon and generates an overall solution by simply concatenating the optimal solutions from these sub-problems. As a “black box” architecture, temporal concatenation works with a wide range of existing MDP algorithms. Our main results characterize the regret of temporal concatenation compared to the optimal solution. We provide upper bounds for general MDP instances, as well as a family of MDP instances in which the upper bounds are shown to be tight. Together, our results demonstrate temporal concatenation's potential of substantial speed-up at the expense of some performance degradation.

An expectation-based account of subject islands and parasitism

2012 ◽  
Vol 49 (2) ◽  
pp. 285-327 ◽  
Author(s):  
RUI P. CHAVES
Keyword(s):  
Full Range ◽  
Search Space ◽  
Functional Explanation ◽  
Parasitic Extraction ◽  
Speed Up

Subject phrases impose particularly strong constraints on extraction. Most research assumes a syntactic account (e.g. Kayne 1983, Chomsky 1986, Rizzi 1990, Lasnik & Saito 1992, Takahashi 1994, Uriagereka 1999), but there are also pragmatic accounts (Erteschik-Shir & Lappin 1979; Van Valin 1986, 1995; Erteschik-Shir 2006, 2007) as well as performance-based approaches (Kluender 2004). In this work I argue that none of these accounts captures the full range of empirical facts, and show that subject and adjunct phrases (phrasal or clausal, finite or otherwise) are by no means impermeable to non-parasitic extraction of nominal, prepositional and adverbial phrases. The present empirical reassessment indicates that the phenomena involving subject and adjunct islands defies the formulation of a general grammatical account. Drawing from insights by Engdahl (1983) and Kluender (2004), I argue that subject island effects have a functional explanation. Independently motivated pragmatic and processing limitations cause subject-internal gaps to be heavily dispreferred, and therefore, extremely infrequent. In turn, this has led to heuristic parsing expectations that preempt subject-internal gaps and therefore speed up processing by pruning the search space of filler–gap dependencies. Such expectations cause processing problems when violated, unless they are dampened by prosodic and pragmatic cues that boost the construction of the correct parse. This account predicts subject islands and their (non-)parasitic exceptions.

Design Optimization Method for MR-Brake Actuators

2011 ◽  
Author(s):  
Ozan G. Erol ◽  
Hakan Gurocak ◽  
Berk Gonenc
Keyword(s):  
Optimal Solution ◽  
Volume Ratio ◽  
Optimization Methods ◽  
Search Space ◽  
Optimization Method ◽  
Friction Torque ◽  
Element Analysis ◽  
Taguchi Design Of Experiments ◽  
Variable Friction ◽  
Electronically Controllable

MR-brakes work by varying viscosity of a magnetorheological (MR) fluid inside the brake. This electronically controllable viscosity leads to variable friction torque generated by the actuator. A properly designed MR-brake can have a high torque-to-volume ratio which is quite desirable for an actuator. However, designing an MR-brake is a complex process as there are many parameters involved in the design which can affect the size and torque output significantly. The contribution of this study is a new design approach that combines the Taguchi design of experiments method with parameterized finite element analysis for optimization. Unlike the typical multivariate optimization methods, this approach can identify the dominant parameters of the design and allows the designer to only explore their interactions during the optimization process. This unique feature reduces the size of the search space and the time it takes to find an optimal solution. It normally takes about a week to design an MR-brake manually. Our interactive method allows the designer to finish the design in about two minutes. In this paper, we first present the details of the MR-brake design problem. This is followed by the details of our new approach. Next, we show how to design an MR-brake using this method. Prototype of a new brake was fabricated. Results of experiments with the prototype brake are very encouraging and are in close agreement with the theoretical performance predictions.

scholarly journals A Novel Method for Economic Dispatch with Across Neighborhood Search: A Case Study in a Provincial Power Grid, China

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-18 ◽  
Author(s):  
Guojiang Xiong ◽  
Jing Zhang ◽  
Xufeng Yuan ◽  
Dongyuan Shi ◽  
Yu He ◽  
...  
Keyword(s):  
Power Grid ◽  
Economic Dispatch ◽  
Optimal Solution ◽  
Inequality Constraints ◽  
Search Space ◽  
Neighborhood Search ◽  
Solution Quality ◽  
Promising Alternative ◽  
Comparison Results ◽  
Population Based Algorithm

Economic dispatch (ED) is of cardinal significance for the power system operation. It is mathematically a typical complex nonlinear multivariable strongly coupled optimization problem with equality and inequality constraints, especially considering the valve-point effects. In order to effectively solve the problem, a simple yet very young and efficient population-based algorithm named across neighborhood search (ANS) is implemented in this paper. In ANS, a group of individuals collaboratively navigate through the search space for obtaining the optimal solution by simultaneously searching the neighborhoods of multiple superior solutions. Four benchmark test cases with diverse complexities and characteristics are firstly employed to comprehensively verify the feasibility and effectiveness of ANS. The experimental and comparison results fully demonstrate the superiority of ANS in terms of the final solution quality, convergence speed, robustness, and statistics. In addition, the sensitivities of ANS to variations of population size and across-search degree are studied. Furthermore, ANS is applied to a practical provincial power grid of China. All the comparison results consistently indicate that ANS is highly competitive and can be used as a promising alternative for ED problems.

scholarly journals Disparity estimation using Graph cuts for road applications

2021 ◽  
Vol 297 ◽  
pp. 01055
Author(s):  
Mohamed El Ansari ◽  
Ilyas El Jaafari ◽  
Lahcen Koutti
Keyword(s):  
Stereo Matching ◽  
Optimal Solution ◽  
Similarity Criterion ◽  
Graph Cuts ◽  
Search Space ◽  
Disparity Estimation ◽  
Stereo Images ◽  
Disparity Map ◽  
Current Frame ◽  
Edge Points

This paper proposes a new edge based stereo matching approach for road applications. The new approach consists in matching the edge points extracted from the input stereo images using temporal constraints. At the current frame, we propose to estimate a disparity range for each image line based on the disparity map of its preceding one. The stereo images are divided into multiple parts according to the estimated disparity ranges. The optimal solution of each part is independently approximated via the state-of-the-art energy minimization approach Graph cuts. The disparity search space at each image part is very small compared to the global one, which improves the results and reduces the execution time. Furthermore, as a similarity criterion between corresponding edge points, we propose a new cost function based on the intensity, the gradient magnitude and gradient orientation. The proposed method has been tested on virtual stereo images, and it has been compared to a recently proposed method and the results are satisfactory.

Sign in / Sign up

Export Citation Format

Share Document