scholarly journals CDE++: Learning Categorical Data Embedding by Enhancing Heterogeneous Feature Value Coupling Relationships

Entropy ◽  
2020 ◽  
Vol 22 (4) ◽  
pp. 391
Author(s):  
Bin Dong ◽  
Songlei Jian ◽  
Ke Zuo
Keyword(s):  
Categorical Data ◽  
Representation Learning ◽  
Learning Performance ◽  
Data Embedding ◽  
Categorical Representation ◽  
Learning Tasks ◽  
Clustering And Classification ◽  
Feature Values ◽  
Low Dimensional ◽  
Heterogeneous Feature

Categorical data are ubiquitous in machine learning tasks, and the representation of categorical data plays an important role in the learning performance. The heterogeneous coupling relationships between features and feature values reflect the characteristics of the real-world categorical data which need to be captured in the representations. The paper proposes an enhanced categorical data embedding method, i.e., CDE++, which captures the heterogeneous feature value coupling relationships into the representations. Based on information theory and the hierarchical couplings defined in our previous work CDE (Categorical Data Embedding by learning hierarchical value coupling), CDE++ adopts mutual information and margin entropy to capture feature couplings and designs a hybrid clustering strategy to capture multiple types of feature value clusters. Moreover, Autoencoder is used to learn non-linear couplings between features and value clusters. The categorical data embeddings generated by CDE++ are low-dimensional numerical vectors which are directly applied to clustering and classification and achieve the best performance comparing with other categorical representation learning methods. Parameter sensitivity and scalability tests are also conducted to demonstrate the superiority of CDE++.

scholarly journals Exponential Family Graph Embeddings

2020 ◽  
Vol 34 (04) ◽  
pp. 3357-3364
Author(s):  
Abdulkadir Celikkanat ◽  
Fragkiskos D. Malliaros
Keyword(s):  
Random Walk ◽  
Exponential Family ◽  
Representation Learning ◽  
Learning Problems ◽  
Interaction Patterns ◽  
Network Representation ◽  
Learning Tasks ◽  
Learning Techniques ◽  
Real World Datasets ◽  
Low Dimensional

Representing networks in a low dimensional latent space is a crucial task with many interesting applications in graph learning problems, such as link prediction and node classification. A widely applied network representation learning paradigm is based on the combination of random walks for sampling context nodes and the traditional Skip-Gram model to capture center-context node relationships. In this paper, we emphasize on exponential family distributions to capture rich interaction patterns between nodes in random walk sequences. We introduce the generic exponential family graph embedding model, that generalizes random walk-based network representation learning techniques to exponential family conditional distributions. We study three particular instances of this model, analyzing their properties and showing their relationship to existing unsupervised learning models. Our experimental evaluation on real-world datasets demonstrates that the proposed techniques outperform well-known baseline methods in two downstream machine learning tasks.

scholarly journals An Effective Multi-Label Feature Selection Model Towards Eliminating Noisy Features

2020 ◽  
Vol 10 (22) ◽  
pp. 8093
Author(s):  
Jun Wang ◽  
Yuanyuan Xu ◽  
Hengpeng Xu ◽  
Zhe Sun ◽  
Zhenglu Yang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Feature Selection ◽  
Learning Performance ◽  
Space Structures ◽  
Learning Tasks ◽  
Feature Spaces ◽  
Selection Approach ◽  
Label Correlations ◽  
Feature Selection Approach ◽  
Low Dimensional

Feature selection has devoted a consistently great amount of effort to dimension reduction for various machine learning tasks. Existing feature selection models focus on selecting the most discriminative features for learning targets. However, this strategy is weak in handling two kinds of features, that is, the irrelevant and redundant ones, which are collectively referred to as noisy features. These features may hamper the construction of optimal low-dimensional subspaces and compromise the learning performance of downstream tasks. In this study, we propose a novel multi-label feature selection approach by embedding label correlations (dubbed ELC) to address these issues. Particularly, we extract label correlations for reliable label space structures and employ them to steer feature selection. In this way, label and feature spaces can be expected to be consistent and noisy features can be effectively eliminated. An extensive experimental evaluation on public benchmarks validated the superiority of ELC.

scholarly journals Embedding-based Representation of Categorical Data by Hierarchical Value Coupling Learning

2017 ◽  
Author(s):  
Songlei Jian ◽  
Longbing Cao ◽  
Guansong Pang ◽  
Kai Lu ◽  
Hang Gao
Keyword(s):  
Categorical Data ◽  
State Of The Art ◽  
Data Representation ◽  
Data Embedding ◽  
Feature Values ◽  
Cluster Values ◽  
Embedding Methods ◽  
Effective Analysis

Learning the representation of categorical data with hierarchical value coupling relationships is very challenging but critical for the effective analysis and learning of such data. This paper proposes a novel coupled unsupervised categorical data representation (CURE) framework and its instantiation, i.e., a coupled data embedding (CDE) method, for representing categorical data by hierarchical value-to-value cluster coupling learning. Unlike existing embedding- and similarity-based representation methods which can capture only a part or none of these complex couplings, CDE explicitly incorporates the hierarchical couplings into its embedding representation. CDE first learns two complementary feature value couplings which are then used to cluster values with different granularities. It further models the couplings in value clusters within the same granularity and with different granularities to embed feature values into a new numerical space with independent dimensions. Substantial experiments show that CDE significantly outperforms three popular unsupervised embedding methods and three state-of-the-art similarity-based representation methods.

scholarly journals Low Dimensional State Representation Learning with Reward-shaped Priors

2021 ◽  
Author(s):  
Nicolo Botteghi ◽  
Ruben Obbink ◽  
Daan Geijs ◽  
Mannes Poel ◽  
Beril Sirmacek ◽  
...  
Keyword(s):  
Representation Learning ◽  
State Representation ◽  
Low Dimensional

scholarly journals A REVIEW OF FEATURE EXTRACTION METHODS ON MACHINE LEARNING

2021 ◽  
Vol 6 (22) ◽  
pp. 51-59
Author(s):  
Mustazzihim Suhaidi ◽  
Rabiah Abdul Kadir ◽  
Sabrina Tiun
Keyword(s):  
Machine Learning ◽  
Feature Extraction ◽  
Feature Selection ◽  
Input Data ◽  
Feature Vector ◽  
Learning Algorithm ◽  
Extraction Methods ◽  
Machine Learning Algorithm ◽  
Learning Tasks ◽  
Low Dimensional

Extracting features from input data is vital for successful classification and machine learning tasks. Classification is the process of declaring an object into one of the predefined categories. Many different feature selection and feature extraction methods exist, and they are being widely used. Feature extraction, obviously, is a transformation of large input data into a low dimensional feature vector, which is an input to classification or a machine learning algorithm. The task of feature extraction has major challenges, which will be discussed in this paper. The challenge is to learn and extract knowledge from text datasets to make correct decisions. The objective of this paper is to give an overview of methods used in feature extraction for various applications, with a dataset containing a collection of texts taken from social media.

scholarly journals Semantic Description of Objects in Images Based on Prototype Theory

2020 ◽  
Author(s):  
Omar Vidal Pino ◽  
Erickson R. Nascimento ◽  
Mario F. M. Campos
Keyword(s):  
Semantic Distance ◽  
Prototype Model ◽  
Prototype Theory ◽  
Semantic Description ◽  
Distinctive Features ◽  
Semantic Classification ◽  
Object Categories ◽  
Clustering And Classification ◽  
Low Dimensional ◽  
Semantic Descriptor

This research aims to build a model for the semantic description of objects based on visual features extracted from images. We introduce a novel semantic description approach inspired by the Prototype Theory. Inspired by the human approach used to represent categories, we propose a novel Computational Prototype Model (CPM) that encodes and stores the object’s image category’s central semantic meaning: the semantic prototype. Our CPM model represents and constructs the semantic prototypes of object categories using Convolutional Neural Networks (CNN). The proposed Prototype-based Description Model uses the CPM model to describe an object highlighting its most distinctive features within the category. Our Global Semantic Descriptor (GSDP) builds discriminative, low-dimensional, and semantically interpretable signatures that encode the objects’ semantic information using the constructed semantic prototypes. It uses the proposed Prototypical Similarity Layer (PS-Layer) to retrieve the category prototype using the principle of categorization based on prototypes. Using different datasets, we show in our experiments that: i) the proposed CPM model successfully simulates the internal semantic structure of the categories; ii) the proposed semantic distance metric can be understood as the object typicality score within a category; iii) our semantic classification method based on prototypes can improve the performance and interpretation of CNN classification models; iv) our semantic descriptor encoding significantly outperforms others state-of-the-art image global encoding in clustering and classification tasks.

scholarly journals Clustering-Based Relational Unsupervised Representation Learning with an Explicit Distributed Representation

2017 ◽  
Author(s):  
Sebastijan Dumancic ◽  
Hendrik Blockeel
Keyword(s):  
Relational Learning ◽  
Representation Learning ◽  
Relational Data ◽  
Distributed Representation ◽  
Learning Tasks ◽  
Wide Range ◽  
Lower Complexity ◽  
Classification Tasks ◽  
Latent Representations

The goal of unsupervised representation learning is to extract a new representation of data, such that solving many different tasks becomes easier. Existing methods typically focus on vectorized data and offer little support for relational data, which additionally describes relationships among instances. In this work we introduce an approach for relational unsupervised representation learning. Viewing a relational dataset as a hypergraph, new features are obtained by clustering vertices and hyperedges. To find a representation suited for many relational learning tasks, a wide range of similarities between relational objects is considered, e.g. feature and structural similarities. We experimentally evaluate the proposed approach and show that models learned on such latent representations perform better, have lower complexity, and outperform the existing approaches on classification tasks.

scholarly journals Masking Orchestration: Multi-Task Pretraining for Multi-Role Dialogue Representation Learning

2020 ◽  
Vol 34 (05) ◽  
pp. 9217-9224
Author(s):  
Tianyi Wang ◽  
Yating Zhang ◽  
Xiaozhong Liu ◽  
Changlong Sun ◽  
Qiong Zhang
Keyword(s):  
Knowledge Integration ◽  
Question Answering ◽  
Representation Learning ◽  
Essential Information ◽  
External Knowledge ◽  
Learning Tasks ◽  
Wide Range ◽  
Specific Learning

Multi-role dialogue understanding comprises a wide range of diverse tasks such as question answering, act classification, dialogue summarization etc. While dialogue corpora are abundantly available, labeled data, for specific learning tasks, can be highly scarce and expensive. In this work, we investigate dialogue context representation learning with various types unsupervised pretraining tasks where the training objectives are given naturally according to the nature of the utterance and the structure of the multi-role conversation. Meanwhile, in order to locate essential information for dialogue summarization/extraction, the pretraining process enables external knowledge integration. The proposed fine-tuned pretraining mechanism is comprehensively evaluated via three different dialogue datasets along with a number of downstream dialogue-mining tasks. Result shows that the proposed pretraining mechanism significantly contributes to all the downstream tasks without discrimination to different encoders.

scholarly journals Neural Excursions from Low-Dimensional Manifold Structure Explain Intersubject Variation in Human Motor Learning

2021 ◽  
Author(s):  
Corson N Areshenkoff ◽  
Daniel J Gale ◽  
Joe Y Nashed ◽  
Dominic Standage ◽  
John Randall Flanagan ◽  
...  
Keyword(s):  
Motor Learning ◽  
Large Scale ◽  
Brain Activity ◽  
Dimensional Subspace ◽  
Learning Performance ◽  
Dimensional Manifold ◽  
Learning Abilities ◽  
Manifold Structure ◽  
Electrophysiological Studies ◽  
Low Dimensional

Humans vary greatly in their motor learning abilities, yet little is known about the neural mechanisms that underlie this variability. Recent neuroimaging and electrophysiological studies demonstrate that large-scale neural dynamics inhabit a low-dimensional subspace or manifold, and that learning is constrained by this intrinsic manifold architecture. Here we asked, using functional MRI, whether subject-level differences in neural excursion from manifold structure can explain differences in learning across participants. We had subjects perform a sensorimotor adaptation task in the MRI scanner on two consecutive days, allowing us to assess their learning performance across days, as well as continuously measure brain activity. We find that the overall neural excursion from manifold activity in both cognitive and sensorimotor brain networks is associated with differences in subjects' patterns of learning and relearning across days. These findings suggest that off-manifold activity provides an index of the relative engagement of different neural systems during learning, and that intersubject differences in patterns of learning and relearning across days are related to reconfiguration processes in cognitive and sensorimotor networks during learning.

Sign in / Sign up

Export Citation Format

Share Document