Theoretical and Empirical Analysis of a Spatial EA Parallel Boosting Algorithm

2018 ◽  
Vol 26 (1) ◽  
pp. 43-66 ◽  
Author(s):  
Uday Kamath ◽  
Carlotta Domeniconi ◽  
Kenneth De Jong
Keyword(s):  
Real World ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Real World Data ◽  
Meta Level ◽  
Meta Learning ◽  
Robustness To Noise ◽  
Boosting Algorithm ◽  
Efficient Learning ◽  
Empirical Analyses

Many real-world problems involve massive amounts of data. Under these circumstances learning algorithms often become prohibitively expensive, making scalability a pressing issue to be addressed. A common approach is to perform sampling to reduce the size of the dataset and enable efficient learning. Alternatively, one customizes learning algorithms to achieve scalability. In either case, the key challenge is to obtain algorithmic efficiency without compromising the quality of the results. In this article we discuss a meta-learning algorithm (PSBML) that combines concepts from spatially structured evolutionary algorithms (SSEAs) with concepts from ensemble and boosting methodologies to achieve the desired scalability property. We present both theoretical and empirical analyses which show that PSBML preserves a critical property of boosting, specifically, convergence to a distribution centered around the margin. We then present additional empirical analyses showing that this meta-level algorithm provides a general and effective framework that can be used in combination with a variety of learning classifiers. We perform extensive experiments to investigate the trade-off achieved between scalability and accuracy, and robustness to noise, on both synthetic and real-world data. These empirical results corroborate our theoretical analysis, and demonstrate the potential of PSBML in achieving scalability without sacrificing accuracy.

Proficient Normalised Fuzzy K-Means With Initial Centroids Methodology

2018 ◽  
Vol 8 (1) ◽  
pp. 42-59
Author(s):  
Deepali Virmani ◽  
Nikita Jain ◽  
Ketan Parikh ◽  
Shefali Upadhyaya ◽  
Abhishek Srivastav
Keyword(s):  
Unsupervised Learning ◽  
Real World ◽  
Learning Algorithms ◽  
Clustering Algorithms ◽  
Real World Data ◽  
World Data ◽  
Clustering Problem ◽  
Time Required ◽  
Selection Of

This article describes how data is relevant and if it can be organized, linked with other data and grouped into a cluster. Clustering is the process of organizing a given set of objects into a set of disjoint groups called clusters. There are a number of clustering algorithms like k-means, k-medoids, normalized k-means, etc. So, the focus remains on efficiency and accuracy of algorithms. The focus is also on the time it takes for clustering and reducing overlapping between clusters. K-means is one of the simplest unsupervised learning algorithms that solves the well-known clustering problem. The k-means algorithm partitions data into K clusters and the centroids are randomly chosen resulting numeric values prohibits it from being used to cluster real world data containing categorical values. Poor selection of initial centroids can result in poor clustering. This article deals with a proposed algorithm which is a variant of k-means with some modifications resulting in better clustering, reduced overlapping and lesser time required for clustering by selecting initial centres in k-means and normalizing the data.

scholarly journals Challenges in benchmarking stream learning algorithms with real-world data

2020 ◽  
Vol 34 (6) ◽  
pp. 1805-1858
Author(s):  
Vinicius M. A. Souza ◽  
Denis M. dos Reis ◽  
André G. Maletzke ◽  
Gustavo E. A. P. A. Batista
Keyword(s):  
Real World ◽  
Learning Algorithms ◽  
Real World Data ◽  
World Data

Performance Assessment of Learning Algorithms on Multi-Domain Data Sets

2018 ◽  
Vol 8 (1) ◽  
pp. 27-41
Author(s):  
Amit Kumar ◽  
Bikash Kanti Sarkar
Keyword(s):  
Data Mining ◽  
Performance Assessment ◽  
Learning Algorithm ◽  
Wide Spectrum ◽  
Learning Algorithms ◽  
Complex Nature ◽  
Data Sets ◽  
Real World Data ◽  
Comparative Performance ◽  
Assessment Of Learning

This article describes how for the last few decades, data mining research has had significant progress in a wide spectrum of applications. Research in prediction of multi-domain data sets is a challenging task due to the imbalanced, voluminous, conflicting, and complex nature of data sets. A learning algorithm is the most important technique for solving these problems. The learning algorithms are widely used for classification purposes. But choosing the learners that perform best for data sets of particular domains is a challenging task in data mining. This article provides a comparative performance assessment of various state-of-the-art learning algorithms over multi-domain data sets to search the effective classifier(s) for a particular domain, e.g., artificial, natural, semi-natural, etc. In the present article, a total of 14 real world data sets are selected from University of California, Irvine (UCI) machine learning repository for conducting experiments using three competent individual learners and their hybrid combinations.

scholarly journals Performance of Machine Learning Algorithms and Diversity in Data

2018 ◽  
Vol 210 ◽  
pp. 04019 ◽  
Author(s):  
Hyontai SUG
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Real World ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Real World Data ◽  
Random Data ◽  
Data Set ◽  
World Data

Recent world events in go games between human and artificial intelligence called AlphaGo showed the big advancement in machine learning technologies. While AlphaGo was trained using real world data, AlphaGo Zero was trained using massive random data, and the fact that AlphaGo Zero won AlphaGo completely revealed that diversity and size in training data is important for better performance for the machine learning algorithms, especially in deep learning algorithms of neural networks. On the other hand, artificial neural networks and decision trees are widely accepted machine learning algorithms because of their robustness in errors and comprehensibility respectively. In this paper in order to prove that diversity and size in data are important factors for better performance of machine learning algorithms empirically, the two representative algorithms are used for experiment. A real world data set called breast tissue was chosen, because the data set consists of real numbers that is very good property for artificial random data generation. The result of the experiment proved the fact that the diversity and size of data are very important factors for better performance.

scholarly journals Online Multitask Relative Similarity Learning

2017 ◽  
Author(s):  
Shuji Hao ◽  
Peilin Zhao ◽  
Yong Liu ◽  
Steven C. H. Hoi ◽  
Chunyan Miao
Keyword(s):  
Real World ◽  
Learning Algorithm ◽  
Learning Problems ◽  
Similarity Function ◽  
Learning Approaches ◽  
Similarity Learning ◽  
Real World Data ◽  
Real World Datasets ◽  
Online Learning Algorithm ◽  
Relative Similarity

Relative similarity learning~(RSL) aims to learn similarity functions from data with relative constraints. Most previous algorithms developed for RSL are batch-based learning approaches which suffer from poor scalability when dealing with real-world data arriving sequentially. These methods are often designed to learn a single similarity function for a specific task. Therefore, they may be sub-optimal to solve multiple task learning problems. To overcome these limitations, we propose a scalable RSL framework named OMTRSL (Online Multi-Task Relative Similarity Learning). Specifically, we first develop a simple yet effective online learning algorithm for multi-task relative similarity learning. Then, we also propose an active learning algorithm to save the labeling cost. The proposed algorithms not only enjoy theoretical guarantee, but also show high efficacy and efficiency in extensive experiments on real-world datasets.

scholarly journals Continuous self-adaptive optimization to learn multi-task multi-agent

2021 ◽  
Author(s):  
Wenqian Liang ◽  
Ji Wang ◽  
Weidong Bao ◽  
Xiaomin Zhu ◽  
Qingyong Wang ◽  
...  
Keyword(s):  
Real World ◽  
Learning Algorithm ◽  
Superior Performance ◽  
Adaptive Optimization ◽  
Single Task ◽  
Multiple Tasks ◽  
Meta Learning ◽  
Gradient Based ◽  
Multi Agent ◽  
Self Adaptive

AbstractMulti-agent reinforcement learning (MARL) methods have shown superior performance to solve a variety of real-world problems focusing on learning distinct policies for individual tasks. These approaches face problems when applied to the non-stationary real-world: agents trained in specialized tasks cannot achieve satisfied generalization performance across multiple tasks; agents have to learn and store specialized policies for individual task and reliable identities of tasks are hardly observable in practice. To address the challenge continuously adapting to multiple tasks in MARL, we formalize the problem into a two-stage curriculum. Single-task policies are learned with MARL approaches, after that we develop a gradient-based Self-Adaptive Meta-Learning algorithm, SAML, that cannot only distill single-task policies into a unified policy but also can facilitate the unified policy to continuously adapt to new incoming tasks. In addition, to validate the continuous adaptation performance on complex task, we extend the widely adopted StarCraft benchmark SMAC and develop a new multi-task multi-agent StarCraft environment, Meta-SMAC, for testing various aspects of continuous adaptation method. Our experiments with a population of agents show that our method enables significantly more efficient adaptation than reactive baselines across different scenarios.

scholarly journals Framework of Intelligent System for Machine Learning Algorithm Selection in Social Sciences

2022 ◽  
pp. 21-28
Author(s):  
Dijana Oreški ◽  
Keyword(s):  
Machine Learning ◽  
Intelligent System ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Algorithm Selection ◽  
Data Set ◽  
Ranking Algorithms ◽  
Meta Learning ◽  
The Right

The ability to generate data has never been as powerful as today when three quintile bytes of data are generated daily. In the field of machine learning, a large number of algorithms have been developed, which can be used for intelligent data analysis and to solve prediction and descriptive problems in different domains. Developed algorithms have different effects on different problems.If one algorithmworks better on one dataset,the same algorithm may work worse on another data set. The reason is that each dataset has different features in terms of local and global characteristics. It is therefore imperative to know intrinsic algorithms behavior on different types of datasets andchoose the right algorithm for the problem solving. To address this problem, this papergives scientific contribution in meta learning field by proposing framework for identifying the specific characteristics of datasets in two domains of social sciences:education and business and develops meta models based on: ranking algorithms, calculating correlation of ranks, developing a multi-criteria model, two-component index and prediction based on machine learning algorithms. Each of the meta models serve as the basis for the development of intelligent system version. Application of such framework should include a comparative analysis of a large number of machine learning algorithms on a large number of datasetsfromsocial sciences.

scholarly journals Physics-aware Spatiotemporal Modules with Auxiliary Tasks for Meta-Learning

2021 ◽  
Author(s):  
Sungyong Seo ◽  
Chuizheng Meng ◽  
Sirisha Rambhatla ◽  
Yan Liu
Keyword(s):  
Real World ◽  
Spatial Information ◽  
Simulated Data ◽  
Superior Performance ◽  
Local Conservation ◽  
Real World Data ◽  
Physical Systems ◽  
Learning Techniques ◽  
Meta Learning ◽  
Fast Adaptation

Modeling the dynamics of real-world physical systems is critical for spatiotemporal prediction tasks, but challenging when data is limited. The scarcity of real-world data and the difficulty in reproducing the data distribution hinder directly applying meta-learning techniques. Although the knowledge of governing partial differential equations (PDE) of the data can be helpful for the fast adaptation to few observations, it is mostly infeasible to exactly find the equation for observations in real-world physical systems. In this work, we propose a framework, physics-aware meta-learning with auxiliary tasks, whose spatial modules incorporate PDE-independent knowledge and temporal modules utilize the generalized features from the spatial modules to be adapted to the limited data, respectively. The framework is inspired by a local conservation law expressed mathematically as a continuity equation and does not require the exact form of governing equation to model the spatiotemporal observations. The proposed method mitigates the need for a large number of real-world tasks for meta-learning by leveraging spatial information in simulated data to meta-initialize the spatial modules. We apply the proposed framework to both synthetic and real-world spatiotemporal prediction tasks and demonstrate its superior performance with limited observations.

scholarly journals Learning at variable attentional load requires cooperation between working memory, meta-learning and attention-augmented reinforcement learning

2020 ◽  
Author(s):  
Thilo Womelsdorf ◽  
Marcus R. Watson ◽  
Paul Tiesinga
Keyword(s):  
Working Memory ◽  
Reinforcement Learning ◽  
Cognitive Flexibility ◽  
Real World ◽  
Learning Strategy ◽  
Prediction Errors ◽  
Negative Prediction ◽  
Attentional Load ◽  
Meta Learning ◽  
Efficient Learning

AbstractFlexible learning of changing reward contingencies can be realized with different strategies. A fast learning strategy involves using working memory of recently rewarded objects to guide choices. A slower learning strategy uses prediction errors to gradually update value expectations to improve choices. How the fast and slow strategies work together in scenarios with real-world stimulus complexity is not well known. Here, we disentangle their relative contributions in rhesus monkeys while they learned the relevance of object features at variable attentional load. We found that learning behavior across six subjects is consistently best predicted with a model combining (i) fast working memory (ii) slower reinforcement learning from differently weighted positive and negative prediction errors, as well as (iii) selective suppression of non-chosen feature values and (iv) a meta-learning mechanism adjusting exploration rates based on a memory trace of recent errors. These mechanisms cooperate differently at low and high attentional loads. While working memory was essential for efficient learning at lower attentional loads, enhanced weighting of negative prediction errors and meta-learning were essential for efficient learning at higher attentional loads. Together, these findings pinpoint a canonical set of learning mechanisms and demonstrate how they cooperate when subjects flexibly adjust to environments with variable real-world attentional demands.Significance statementLearning which visual features are relevant for achieving our goals is challenging in real-world scenarios with multiple distracting features and feature dimensions. It is known that in such scenarios learning benefits significantly from attentional prioritization. Here we show that beyond attention, flexible learning uses a working memory system, a separate learning gain for avoiding negative outcomes, and a meta-learning process that adaptively increases exploration rates whenever errors accumulate. These subcomponent processes of cognitive flexibility depend on distinct learning signals that operate at varying timescales, including the most recent reward outcome (for working memory), memories of recent outcomes (for adjusting exploration), and reward prediction errors (for attention augmented reinforcement learning). These results illustrate the specific mechanisms that cooperate during cognitive flexibility.

Sign in / Sign up

Export Citation Format

Share Document