scholarly journals MEDA: Meta-Learning with Data Augmentation for Few-Shot Text Classification

2021 ◽  
Author(s):  
Pengfei Sun ◽  
Yawen Ouyang ◽  
Wenming Zhang ◽  
Xin-yu Dai
Keyword(s):  
Text Classification ◽  
Data Augmentation ◽  
State Of The Art ◽  
Promising Technique ◽  
Learning Methods ◽  
Text Data ◽  
Art Methods ◽  
Visual Tasks ◽  
Meta Learning

Meta-learning has recently emerged as a promising technique to address the challenge of few-shot learning. However, standard meta-learning methods mainly focus on visual tasks, which makes it hard for them to deal with diverse text data directly. In this paper, we introduce a novel framework for few-shot text classification, which is named as MEta-learning with Data Augmentation (MEDA). MEDA is composed of two modules, a ball generator and a meta-learner, which are learned jointly. The ball generator is to increase the number of shots per class by generating more samples, so that meta-learner can be trained with both original and augmented samples. It is worth noting that ball generator is agnostic to the choice of the meta-learning methods. Experiment results show that on both datasets, MEDA outperforms existing state-of-the-art methods and significantly improves the performance of meta-learning on few-shot text classification.

Deep Learning for text in limted data settings

2020 ◽  
Author(s):  
Pathikkumar Patel ◽  
Bhargav Lad ◽  
Jinan Fiaidhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
Time Series Forecasting ◽  
Text Data ◽  
Performance Levels

During the last few years, RNN models have been extensively used and they have proven to be better for sequence and text data. RNNs have achieved state-of-the-art performance levels in several applications such as text classification, sequence to sequence modelling and time series forecasting. In this article we will review different Machine Learning and Deep Learning based approaches for text data and look at the results obtained from these methods. This work also explores the use of transfer learning in NLP and how it affects the performance of models on a specific application of sentiment analysis.

scholarly journals Sentiment Classification Using Convolutional Neural Networks

2019 ◽  
Vol 9 (11) ◽  
pp. 2347 ◽  
Author(s):  
Hannah Kim ◽  
Young-Seob Jeong
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Learning Models ◽  
Text Data ◽  
Textual Data ◽  
Better Than

As the number of textual data is exponentially increasing, it becomes more important to develop models to analyze the text data automatically. The texts may contain various labels such as gender, age, country, sentiment, and so forth. Using such labels may bring benefits to some industrial fields, so many studies of text classification have appeared. Recently, the Convolutional Neural Network (CNN) has been adopted for the task of text classification and has shown quite successful results. In this paper, we propose convolutional neural networks for the task of sentiment classification. Through experiments with three well-known datasets, we show that employing consecutive convolutional layers is effective for relatively longer texts, and our networks are better than other state-of-the-art deep learning models.

scholarly journals Effectiveness, Explainability and Reliability of Machine Meta-Learning Methods for Predicting Mortality in Patients with COVID-19: Results of the Brazilian COVID-19 Registry

2021 ◽  
Author(s):  
Bruno Barbosa Miranda de Paiva ◽  
Polianna Delfino Pereira ◽  
Claudio Moises Valiense de Andrade ◽  
Virginia Mara Reis Gomes ◽  
Maria Clara Pontello Barbosa Lima ◽  
...  
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
State Of The Art ◽  
Laboratory Data ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Learning Models ◽  
Learning Methods ◽  
Meta Learning ◽  
Machine Learning Models

Objective: To provide a thorough comparative study among state ofthe art machine learning methods and statistical methods for determining in-hospital mortality in COVID 19 patients using data upon hospital admission; to study the reliability of the predictions of the most effective methods by correlating the probability of the outcome and the accuracy of the methods; to investigate how explainable are the predictions produced by the most effective methods. Materials and Methods: De-identified data were obtained from COVID 19 positive patients in 36 participating hospitals, from March 1 to September 30, 2020. Demographic, comorbidity, clinical presentation and laboratory data were used as training data to develop COVID 19 mortality prediction models. Multiple machine learning and traditional statistics models were trained on this prediction task using a folded cross validation procedure, from which we assessed performance and interpretability metrics. Results: The Stacking of machine learning models improved over the previous state of the art results by more than 26% in predicting the class of interest (death), achieving 87.1% of AUROC and macroF1 of 73.9%. We also show that some machine learning models can be very interpretable and reliable, yielding more accurate predictions while providing a good explanation for the why. Conclusion: The best results were obtained using the meta learning ensemble model Stacking. State of the art explainability techniques such as SHAP values can be used to draw useful insights into the patterns learned by machine-learning algorithms. Machine learning models can be more explainable than traditional statistics models while also yielding highly reliable predictions. Key words: COVID-19; prognosis; prediction model; machine learning

Use of Distributed Semi-Supervised Clustering for Text Classification

2019 ◽  
Vol 28 (08) ◽  
pp. 1950127 ◽  
Author(s):  
Pei Li ◽  
Ze Deng
Keyword(s):  
Text Classification ◽  
Large Scale ◽  
State Of The Art ◽  
Main Stream ◽  
Text Data ◽  
Supervised Clustering ◽  
Text Collections ◽  
Textual Data ◽  
Clustering Approach ◽  
Computing Performance

Text classification is an important way to handle and organize textual data. Among existing methods of text classification, semi-supervised clustering is a main-stream technique. In the era of ‘Big data’, the current semi-supervised clustering approaches for text classification generally do not apply for excessive costs in scalability and computing performance for massive text data. Aiming at this problem, this study proposes a scalable text classification algorithm for large-scale text collections, namely D-TESC by modifying a state-of-the-art semi-supervised clustering approach for text classification in a centralized fashion (TESC). D-TESC can process the textual data in a distributed manner to meet a great scalability. The experimental results indicate that (1) the D-TESC algorithm has a comparable classification quality with TESC, and (2) outperforms TESC by average 7.2 times by using eight CPU threads in terms of scalability.

scholarly journals When Low Resource NLP Meets Unsupervised Language Model: Meta-Pretraining then Meta-Learning for Few-Shot Text Classification (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13773-13774
Author(s):  
Shumin Deng ◽  
Ningyu Zhang ◽  
Zhanlin Sun ◽  
Jiaoyan Chen ◽  
Huajun Chen
Keyword(s):  
Text Classification ◽  
State Of The Art ◽  
Language Model ◽  
Language Models ◽  
Generic Model ◽  
Effective Strategy ◽  
Linguistic Features ◽  
Meta Learning ◽  
Promising Solution ◽  
Model Initialization

Text classification tends to be difficult when data are deficient or when it is required to adapt to unseen classes. In such challenging scenarios, recent studies have often used meta-learning to simulate the few-shot task, thus negating implicit common linguistic features across tasks. This paper addresses such problems using meta-learning and unsupervised language models. Our approach is based on the insight that having a good generalization from a few examples relies on both a generic model initialization and an effective strategy for adapting this model to newly arising tasks. We show that our approach is not only simple but also produces a state-of-the-art performance on a well-studied sentiment classification dataset. It can thus be further suggested that pretraining could be a promising solution for few-shot learning of many other NLP tasks. The code and the dataset to replicate the experiments are made available at https://github.com/zxlzr/FewShotNLP.

scholarly journals Application of the Polyhedral Conic Functions Method in the Text Classification and Comparative Analysis

2018 ◽  
Vol 2018 ◽  
pp. 1-11
Author(s):  
Nur Uylaş Satı ◽  
Burak Ordin
Keyword(s):  
Text Classification ◽  
Numerical Experiments ◽  
Text Categorization ◽  
Cross Validation ◽  
State Of The Art ◽  
Inequality Constraints ◽  
Online Information ◽  
Direct Proportion ◽  
Art Methods ◽  
Tenfold Cross Validation

In direct proportion to the heavy increase of online information data, the attention to text categorization (classification) has also increased. In text categorization problem, namely, text classification, the goal is to classify the documents into predefined classes (categories or labels). Recently various methods in data mining have been experienced for text classification in literature except polyhedral conic function (PCF) methods. In this paper, PCFs are used to classify the documents. The separation algorithms via PCFs which include linear programming subproblems with inequality constraints are presented. Numerical experiments are done on real-world text datasets. Comparisons are made between state-of-the-art methods by presenting obtained tenfold cross-validation results, accuracy values, and running times in tables. The results verify that in text classification PCF methods are as effective in terms of accuracy values as state-of-the-art methods.

scholarly journals An Efficient Multi-Scale Focusing Attention Network for Person Re-Identification

2021 ◽  
Vol 11 (5) ◽  
pp. 2010
Author(s):  
Wei Huang ◽  
Yongying Li ◽  
Kunlin Zhang ◽  
Xiaoyu Hou ◽  
Jihui Xu ◽  
...  
Keyword(s):  
Computational Complexity ◽  
Data Augmentation ◽  
High Efficiency ◽  
State Of The Art ◽  
Computational Cost ◽  
Attention Network ◽  
Attention Model ◽  
Multi Scale ◽  
Model Generalization ◽  
Art Methods

The multi-scale lightweight network and attention mechanism recently attracted attention in person re-identification (ReID) as it is capable of improving the model’s ability to process information with low computational cost. However, state-of-the-art methods mostly concentrate on the spatial attention and big block channel attention model with high computational complexity while rarely investigate the inside block attention with the lightweight network, which cannot meet the requirements of high efficiency and low latency in the actual ReID system. In this paper, a novel lightweight person ReID model is designed firstly, called Multi-Scale Focusing Attention Network (MSFANet), to capture robust and elaborate multi-scale ReID features, which have fewer float-computing and higher performance. MSFANet is achieved by designing a multi-branch depthwise separable convolution module, combining with an inside block attention module, to extract and fuse multi-scale features independently. In addition, we design a multi-stage backbone with the ‘1-2-3’ form, which can significantly reduce computational cost. Furthermore, the MSFANet is exceptionally lightweight and can be embedded in a ReID framework flexibly. Secondly, an efficient loss function combining softmax loss and TriHard loss, based on the proposed optimal data augmentation method, is designed for faster convergence and better model generalization ability. Finally, the experimental results on two big ReID datasets (Market1501 and DukeMTMC) and two small ReID datasets (VIPeR, GRID) show that the proposed MSFANet achieves the best mAP performance and the lowest computational complexity compared with state-of-the-art methods, which are increasing by 2.3% and decreasing by 18.2%, respectively.

scholarly journals Do Not Have Enough Data? Deep Learning to the Rescue!

2020 ◽  
Vol 34 (05) ◽  
pp. 7383-7390 ◽  
Author(s):  
Ateret Anaby-Tavor ◽  
Boaz Carmeli ◽  
Esther Goldbraich ◽  
Amir Kantor ◽  
George Kour ◽  
...  
Keyword(s):  
Text Classification ◽  
Data Augmentation ◽  
State Of The Art ◽  
Language Model ◽  
Original Data ◽  
Fine Tuning ◽  
Initial Training ◽  
Series Of Experiments ◽  
Classification Tasks ◽  
Trained Neural Network

Based on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks. We use a powerful pre-trained neural network model to artificially synthesize new labeled data for supervised learning. We mainly focus on cases with scarce labeled data. Our method, referred to as language-model-based data augmentation (LAMBADA), involves fine-tuning a state-of-the-art language generator to a specific task through an initial training phase on the existing (usually small) labeled data. Using the fine-tuned model and given a class label, new sentences for the class are generated. Our process then filters these new sentences by using a classifier trained on the original data. In a series of experiments, we show that LAMBADA improves classifiers' performance on a variety of datasets. Moreover, LAMBADA significantly improves upon the state-of-the-art techniques for data augmentation, specifically those applicable to text classification tasks with little data.

scholarly journals Active semi-supervised framework with data editing

2012 ◽  
Vol 9 (4) ◽  
pp. 1513-1532 ◽  
Author(s):  
Xue Zhang ◽  
Wangxin Xiao
Keyword(s):  
Active Learning ◽  
Supervised Learning ◽  
Text Classification ◽  
State Of The Art ◽  
The Self ◽  
Training Data ◽  
Data Sets ◽  
Text Data ◽  
Data Editing ◽  
Data Problem

In order to address the insufficient training data problem, many active semi-supervised algorithms have been proposed. The self-labeled training data in semi-supervised learning may contain much noise due to the insufficient training data. Such noise may snowball themselves in the following learning process and thus hurt the generalization ability of the final hypothesis. Extremely few labeled training data in sparsely labeled text classification aggravate such situation. If such noise could be identified and removed by some strategy, the performance of the active semi-supervised algorithms should be improved. However, such useful techniques of identifying and removing noise have been seldom explored in existing active semi-supervised algorithms. In this paper, we propose an active semi-supervised framework with data editing (we call it ASSDE) to improve sparsely labeled text classification. A data editing technique is used to identify and remove noise introduced by semi-supervised labeling. We carry out the data editing technique by fully utilizing the advantage of active learning, which is novel according to our knowledge. The fusion of active learning with data editing makes ASSDE more robust to the sparsity and the distribution bias of the training data. It further simplifies the design of semi-supervised learning which makes ASSDE more efficient. Extensive experimental study on several real-world text data sets shows the encouraging results of the proposed framework for sparsely labeled text classification, compared with several state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document