scholarly journals A Large-Scale Experimental Evaluation of High-Performing Multi- and Many-Objective Evolutionary Algorithms

2018 ◽  
Vol 26 (4) ◽  
pp. 621-656 ◽  
Author(s):  
Leonardo C. T. Bezerra ◽  
Manuel López-Ibáñez ◽  
Thomas Stützle
Keyword(s):  
Evolutionary Algorithms ◽  
Large Scale ◽  
Performance Metrics ◽  
Comprehensive Evaluation ◽  
Assessment Tools ◽  
Apparent Contradiction ◽  
Multi Objective ◽  
Automatic Algorithm Configuration ◽  
Wide Range ◽  
Evolutionary Operators

Research on multi-objective evolutionary algorithms (MOEAs) has produced over the past decades a large number of algorithms and a rich literature on performance assessment tools to evaluate and compare them. Yet, newly proposed MOEAs are typically compared against very few, often a decade older MOEAs. One reason for this apparent contradiction is the lack of a common baseline for comparison, with each subsequent study often devising its own experimental scenario, slightly different from other studies. As a result, the state of the art in MOEAs is a disputed topic. This article reports a systematic, comprehensive evaluation of a large number of MOEAs that covers a wide range of experimental scenarios. A novelty of this study is the separation between the higher-level algorithmic components related to multi-objective optimization (MO), which characterize each particular MOEA, and the underlying parameters—such as evolutionary operators, population size, etc.—whose configuration may be tuned for each scenario. Instead of relying on a common or “default” parameter configuration that may be low-performing for particular MOEAs or scenarios and unintentionally biased, we tune the parameters of each MOEA for each scenario using automatic algorithm configuration methods. Our results confirm some of the assumed knowledge in the field, while at the same time they provide new insights on the relative performance of MOEAs for many-objective problems. For example, under certain conditions, indicator-based MOEAs are more competitive for such problems than previously assumed. We also analyze problem-specific features affecting performance, the agreement between performance metrics, and the improvement of tuned configurations over the default configurations used in the literature. Finally, the data produced is made publicly available to motivate further analysis and a baseline for future comparisons.

scholarly journals Design and Evaluation of Large-Scale IoT-Enabled Healthcare Architecture

2021 ◽  
Vol 11 (8) ◽  
pp. 3623
Author(s):  
Omar Said ◽  
Amr Tolba
Keyword(s):  
Packet Loss ◽  
Large Scale ◽  
Performance Metrics ◽  
Medical System ◽  
Healthcare Data ◽  
High Altitude Platforms ◽  
Wide Range ◽  
Simulation Results ◽  
Healthcare Architecture ◽  
The Internet Of Things

Employment of the Internet of Things (IoT) technology in the healthcare field can contribute to recruiting heterogeneous medical devices and creating smart cooperation between them. This cooperation leads to an increase in the efficiency of the entire medical system, thus accelerating the diagnosis and curing of patients, in general, and rescuing critical cases in particular. In this paper, a large-scale IoT-enabled healthcare architecture is proposed. To achieve a wide range of communication between healthcare devices, not only are Internet coverage tools utilized but also satellites and high-altitude platforms (HAPs). In addition, the clustering idea is applied in the proposed architecture to facilitate its management. Moreover, healthcare data are prioritized into several levels of importance. Finally, NS3 is used to measure the performance of the proposed IoT-enabled healthcare architecture. The performance metrics are delay, energy consumption, packet loss, coverage tool usage, throughput, percentage of served users, and percentage of each exchanged data type. The simulation results demonstrate that the proposed IoT-enabled healthcare architecture outperforms the traditional healthcare architecture.

CMIP model evaluation with the ESMValTool v2.0

2020 ◽  
Author(s):  
Axel Lauer ◽  
Fernando Iglesias-Suarez ◽  
Veronika Eyring ◽  
the ESMValTool development team
Keyword(s):  
Model Evaluation ◽  
Large Scale ◽  
Performance Metrics ◽  
Comprehensive Evaluation ◽  
Coupled Model ◽  
Data Sets ◽  
Evaluation Tool ◽  
Emergent Constraints ◽  
Data Volume ◽  
And Performance

<p>The Earth System Model Evaluation Tool (ESMValTool) has been developed with the aim of taking model evaluation to the next level by facilitating analysis of many different ESM components, providing well-documented source code and scientific background of implemented diagnostics and metrics and allowing for traceability and reproducibility of results (provenance). This has been made possible by a lively and growing development community continuously improving the tool supported by multiple national and European projects. The latest version (2.0) of the ESMValTool has been developed as a large community effort to specifically target the increased data volume of the Coupled Model Intercomparison Project Phase 6 (CMIP6) and the related challenges posed by analysis and evaluation of output from multiple high-resolution and complex ESMs. For this, the core functionalities have been completely rewritten in order to take advantage of state-of-the-art computational libraries and methods to allow for efficient and user-friendly data processing. Common operations on the input data such as regridding or computation of multi-model statistics are now centralized in a highly optimized preprocessor written in Python. The diagnostic part of the ESMValTool includes a large collection of standard recipes for reproducing peer-reviewed analyses of many variables across atmosphere, ocean, and land domains, with diagnostics and performance metrics focusing on the mean-state, trends, variability and important processes, phenomena, as well as emergent constraints. While most of the diagnostics use observational data sets (in particular satellite and ground-based observations) or reanalysis products for model evaluation some are also based on model-to-model comparisons. This presentation introduces the diagnostics newly implemented into ESMValTool v2.0 including an extended set of large-scale diagnostics for quasi-operational and comprehensive evaluation of ESMs, new diagnostics for extreme events, regional model and impact evaluation and analysis of ESMs, as well as diagnostics for emergent constraints and analysis of future projections from ESMs. The new diagnostics are illustrated with examples using results from the well-established CMIP5 and the newly available CMIP6 data sets.</p>

Multi-Objective Evolutionary Algorithms

2016 ◽  
pp. 301-345
Author(s):  
A. K. Nandi ◽  
K. Deb
Keyword(s):  
Genetic Algorithm ◽  
Evolutionary Algorithms ◽  
Optimization Problem ◽  
Performance Metrics ◽  
Research Work ◽  
Primary Objective ◽  
Multi Objective Optimization ◽  
Nsga Ii ◽  
Mould Material ◽  
Multi Objective

The primary objective in designing appropriate particle reinforced polyurethane composite which will be used as a mould material in soft tooling process is to minimize the cycle time of soft tooling process by providing faster cooling rate during solidification of wax/plastic component. This chapter exemplifies an effective approach to design particle reinforced mould materials by solving the inherent multi-objective optimization problem associated with soft tooling process using evolutionary algorithms. In this chapter, first a brief introduction of multi-objective optimization problem with the key issues is presented. Then, after a short overview on the working procedure of genetic algorithm, a well- established multi-objective evolutionary algorithm, namely NSGA-II along with various performance metrics are described. The inherent multi-objective problem in soft tooling process is demonstrated and subsequently solved using an elitist non-dominated sorting genetic algorithm, NSGA-II. Multi-objective optimization results obtained using NSGA-II are analyzed statistically and validated with real industrial application. Finally the fundamental results of this approach are summarized and various perspectives to the industries along with scopes for future research work are pointed out.

Efficient Portfolio Construction with the Use of Multiobjective Evolutionary Algorithms: Best Practices and Performance Metrics

2015 ◽  
Vol 14 (03) ◽  
pp. 535-564 ◽  
Author(s):  
K. Liagkouras ◽  
K. Metaxiotis
Keyword(s):  
Best Practices ◽  
Evolutionary Algorithms ◽  
Portfolio Optimization ◽  
Optimization Problem ◽  
Performance Metrics ◽  
State Of The Art ◽  
The State ◽  
Multiobjective Evolutionary Algorithms ◽  
Wide Range ◽  
Portfolio Optimization Problem

This paper provides a systematic study of the technologies and algorithms associated with the implementation of multiobjective evolutionary algorithms (MOEAs) for the solution of the portfolio optimization problem. Based on the examination of the state-of-the art we provide the best practices for dealing with the complexities of the constrained portfolio optimization problem (CPOP). In particular, rigorous algorithmic and technical treatment is provided for the efficient incorporation of a wide range of real-world constraints into the MOEAs. Moreover, we address special configuration issues related to the application of MOEAs for solving the CPOP. Finally, by examining the state-of-the-art we identify the most appropriate performance metrics for the evaluation of the relevant results from the implementation of the MOEAs to the solution of the CPOP.

scholarly journals Automatically Designing State-of-the-Art Multi- and Many-Objective Evolutionary Algorithms

2020 ◽  
Vol 28 (2) ◽  
pp. 195-226 ◽  
Author(s):  
Leonardo C. T. Bezerra ◽  
Manuel López-Ibáñez ◽  
Thomas Stützle
Keyword(s):  
Evolutionary Algorithms ◽  
Performance Metrics ◽  
State Of The Art ◽  
Continuous Optimization ◽  
Parameter Tuning ◽  
High Performing ◽  
Performance Metric ◽  
Wide Range ◽  
Algorithmic Framework ◽  
Art Performance

A recent comparison of well-established multiobjective evolutionary algorithms (MOEAs) has helped better identify the current state-of-the-art by considering (i) parameter tuning through automatic configuration, (ii) a wide range of different setups, and (iii) various performance metrics. Here, we automatically devise MOEAs with verified state-of-the-art performance for multi- and many-objective continuous optimization. Our work is based on two main considerations. The first is that high-performing algorithms can be obtained from a configurable algorithmic framework in an automated way. The second is that multiple performance metrics may be required to guide this automatic design process. In the first part of this work, we extend our previously proposed algorithmic framework, increasing the number of MOEAs, underlying evolutionary algorithms, and search paradigms that it comprises. These components can be combined following a general MOEA template, and an automatic configuration method is used to instantiate high-performing MOEA designs that optimize a given performance metric and present state-of-the-art performance. In the second part, we propose a multiobjective formulation for the automatic MOEA design, which proves critical for the context of many-objective optimization due to the disagreement of established performance metrics. Our proposed formulation leads to an automatically designed MOEA that presents state-of-the-art performance according to a set of metrics, rather than a single one.

Multi-Objective Evolutionary Algorithms

2017 ◽  
pp. 185-229
Author(s):  
A. K. Nandi ◽  
K. Deb
Keyword(s):  
Genetic Algorithm ◽  
Evolutionary Algorithms ◽  
Optimization Problem ◽  
Performance Metrics ◽  
Research Work ◽  
Primary Objective ◽  
Multi Objective Optimization ◽  
Nsga Ii ◽  
Mould Material ◽  
Multi Objective

The primary objective in designing appropriate particle reinforced polyurethane composite which will be used as a mould material in soft tooling process is to minimize the cycle time of soft tooling process by providing faster cooling rate during solidification of wax/plastic component. This chapter exemplifies an effective approach to design particle reinforced mould materials by solving the inherent multi-objective optimization problem associated with soft tooling process using evolutionary algorithms. In this chapter, first a brief introduction of multi-objective optimization problem with the key issues is presented. Then, after a short overview on the working procedure of genetic algorithm, a well- established multi-objective evolutionary algorithm, namely NSGA-II along with various performance metrics are described. The inherent multi-objective problem in soft tooling process is demonstrated and subsequently solved using an elitist non-dominated sorting genetic algorithm, NSGA-II. Multi-objective optimization results obtained using NSGA-II are analyzed statistically and validated with real industrial application. Finally the fundamental results of this approach are summarized and various perspectives to the industries along with scopes for future research work are pointed out.

scholarly journals Performance Evaluation Metrics for Multi-Objective Evolutionary Algorithms in Search-Based Software Engineering: Systematic Literature Review

2021 ◽  
Vol 11 (7) ◽  
pp. 3117
Author(s):  
Jamal Abdullahi Nuh ◽  
Tieng Wei Koh ◽  
Salmi Baharom ◽  
Mohd Hafeez Osman ◽  
Si Na Kew
Keyword(s):  
Systematic Review ◽  
Software Engineering ◽  
Evolutionary Algorithms ◽  
Software Verification ◽  
Performance Metrics ◽  
Scientific Evidence ◽  
Engineering Problem ◽  
Software Project ◽  
Search Based Software Engineering ◽  
Multi Objective

Many recent studies have shown that various multi-objective evolutionary algorithms have been widely applied in the field of search-based software engineering (SBSE) for optimal solutions. Most of them either focused on solving newly re-formulated problems or on proposing new approaches, while a number of studies performed reviews and comparative studies on the performance of proposed algorithms. To evaluate such performance, it is necessary to consider a number of performance metrics that play important roles during the evaluation and comparison of investigated algorithms based on their best-simulated results. While there are hundreds of performance metrics in the literature that can quantify in performing such tasks, there is a lack of systematic review conducted to provide evidence of using these performance metrics, particularly in the software engineering problem domain. In this paper, we aimed to review and quantify the type of performance metrics, number of objectives, and applied areas in software engineering that reported in primary studies—this will eventually lead to inspiring the SBSE community to further explore such approaches in depth. To perform this task, a formal systematic review protocol was applied for planning, searching, and extracting the desired elements from the studies. After considering all the relevant inclusion and exclusion criteria for the searching process, 105 relevant articles were identified from the targeted online databases as scientific evidence to answer the eight research questions. The preliminary results show that remarkable studies were reported without considering performance metrics for the purpose of algorithm evaluation. Based on the 27 performance metrics that were identified, hypervolume, inverted generational distance, generational distance, and hypercube-based diversity metrics appear to be widely adopted in most of the studies in software requirements engineering, software design, software project management, software testing, and software verification. Additionally, there are increasing interest in the community in re-formulating many objective problems with more than three objectives, yet, currently are dominated in re-formulating two to three objectives.

scholarly journals ISORROPIA II: a computationally efficient thermodynamic equilibrium model for K<sup>+</sup>–Ca<sup>2+</sup>–Mg<sup>2+</sup>–NH<sub>4</sub><sup>+</sup>–Na<sup>+</sup>–SO<sub>4</sub><sup>2−</sup>–NO<sub>3</sub><sup>−</sup>–Cl<sup>−</sup>–H<sub>2</sub>O aerosols

2007 ◽  
Vol 7 (1) ◽  
pp. 1893-1939 ◽  
Author(s):  
C. Fountoukis ◽  
A. Nenes
Keyword(s):  
Thermodynamic Equilibrium ◽  
Equilibrium Model ◽  
Large Scale ◽  
Comprehensive Evaluation ◽  
Atmospheric Transport ◽  
Attractive Alternative ◽  
Computationally Efficient ◽  
Calculation Algorithm ◽  
Wide Range ◽  
Thermodynamic Equilibrium Model

Abstract. This study presents ISORROPIA II, a thermodynamic equilibrium model for the K+–Ca2+–Mg2+–NH4+–Na+–SO42−–NO3−–Cl−–H2O aerosol system. A comprehensive evaluation of its performance is conducted against the thermodynamic module SCAPE2 over a wide range of atmospherically relevant conditions. The two models overall agree well, to within 13% for aerosol water content and total PM mass, 16% for aerosol nitrate and 6% for aerosol chloride and ammonium. Largest discrepancies were found under conditions of low RH, primarily from differences in the treatment of water uptake and solid state composition. In terms of computational speed, ISORROPIA II was always found to be more than an order of magnitude faster than SCAPE2, with robust and rapid convergence under all conditions. The addition of crustal species does not slow down the thermodynamic calculations (compared to the older ISORROPIA code) because of optimizations in the activity coefficient calculation algorithm. Based on its computational rigor and performance, ISORROPIA II appears to be a highly attractive alternative for use in large scale air quality and atmospheric transport models.

Sign in / Sign up

Export Citation Format

Share Document