scholarly journals From Machine Translated NLI Corpus to Universal Sentence Representations in Czech

2020 ◽  
Author(s):  
Martin Víta
Keyword(s):  
Universal Sentence

Adversarial training for few-shot text classification

2021 ◽  
Vol 14 (2) ◽  
pp. 201-214
Author(s):  
Danilo Croce ◽  
Giuseppe Castellucci ◽  
Roberto Basili
Keyword(s):  
Supervised Learning ◽  
Language Processing ◽  
Reproducing Kernel ◽  
Generative Adversarial Networks ◽  
Training Material ◽  
Semantic Classification ◽  
Universal Sentence ◽  
Kernel Hilbert Spaces ◽  
Supervised Methods ◽  
Low Dimensional

In recent years, Deep Learning methods have become very popular in classification tasks for Natural Language Processing (NLP); this is mainly due to their ability to reach high performances by relying on very simple input representations, i.e., raw tokens. One of the drawbacks of deep architectures is the large amount of annotated data required for an effective training. Usually, in Machine Learning this problem is mitigated by the usage of semi-supervised methods or, more recently, by using Transfer Learning, in the context of deep architectures. One recent promising method to enable semi-supervised learning in deep architectures has been formalized within Semi-Supervised Generative Adversarial Networks (SS-GANs) in the context of Computer Vision. In this paper, we adopt the SS-GAN framework to enable semi-supervised learning in the context of NLP. We demonstrate how an SS-GAN can boost the performances of simple architectures when operating in expressive low-dimensional embeddings; these are derived by combining the unsupervised approximation of linguistic Reproducing Kernel Hilbert Spaces and the so-called Universal Sentence Encoders. We experimentally evaluate the proposed approach over a semantic classification task, i.e., Question Classification, by considering different sizes of training material and different numbers of target classes. By applying such adversarial schema to a simple Multi-Layer Perceptron, a classifier trained over a subset derived from 1% of the original training material achieves 92% of accuracy. Moreover, when considering a complex classification schema, e.g., involving 50 classes, the proposed method outperforms state-of-the-art alternatives such as BERT.

Machine-learning as a validated tool to characterize individual differences in free recall of naturalistic events.

2021 ◽  
Author(s):  
Xinxu Shen ◽  
Troy Houser ◽  
David Victor Smith ◽  
Vishnu P. Murty
Keyword(s):  
Machine Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Individual Difference ◽  
Language Processing ◽  
Large Scale ◽  
High Reliability ◽  
Difference Analysis ◽  
Universal Sentence ◽  
Natural Language Processing Tool

The use of naturalistic stimuli, such as narrative movies, is gaining popularity in many fields, characterizing memory, affect, and decision-making. Narrative recall paradigms are often used to capture the complexity and richness of memory for naturalistic events. However, scoring narrative recalls is time-consuming and prone to human biases. Here, we show the validity and reliability of using a natural language processing tool, the Universal Sentence Encoder (USE), to automatically score narrative recall. We compared the reliability in scoring made between two independent raters (i.e., hand-scored) and between our automated algorithm and individual raters (i.e., automated) on trial-unique, video clips of magic tricks. Study 1 showed that our automated segmentation approaches yielded high reliability and reflected measures yielded by hand-scoring, and further that the results using USE outperformed another popular natural language processing tool, GloVe. In study two, we tested whether our automated approach remained valid when testing individual’s varying on clinically-relevant dimensions that influence episodic memory, age and anxiety. We found that our automated approach was equally reliable across both age groups and anxiety groups, which shows the efficacy of our approach to assess narrative recall in large-scale individual difference analysis. In sum, these findings suggested that machine learning approaches implementing USE are a promising tool for scoring large-scale narrative recalls and perform individual difference analysis for research using naturalistic stimuli.

scholarly journals Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study (Preprint)

2019 ◽  
Author(s):  
Derek Howard ◽  
Marta M Maslej ◽  
Justin Lee ◽  
Jacob Ritchie ◽  
Geoffrey Woollard ◽  
...  
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Social Media ◽  
Transfer Learning ◽  
Computational Linguistics ◽  
Feature Representation ◽  
Fine Tuning ◽  
Language Models ◽  
Universal Sentence ◽  
Text Feature

BACKGROUND Mental illness affects a significant portion of the worldwide population. Online mental health forums can provide a supportive environment for those afflicted and also generate a large amount of data that can be mined to predict mental health states using machine learning methods. OBJECTIVE This study aimed to benchmark multiple methods of text feature representation for social media posts and compare their downstream use with automated machine learning (AutoML) tools. We tested on datasets that contain posts labeled for perceived suicide risk or moderator attention in the context of self-harm. Specifically, we assessed the ability of the methods to prioritize posts that a moderator would identify for immediate response. METHODS We used 1588 labeled posts from the Computational Linguistics and Clinical Psychology (CLPsych) 2017 shared task collected from the Reachout.com forum. Posts were represented using lexicon-based tools, including Valence Aware Dictionary and sEntiment Reasoner, Empath, and Linguistic Inquiry and Word Count, and also using pretrained artificial neural network models, including DeepMoji, Universal Sentence Encoder, and Generative Pretrained Transformer-1 (GPT-1). We used Tree-based Optimization Tool and Auto-Sklearn as AutoML tools to generate classifiers to triage the posts. RESULTS The top-performing system used features derived from the GPT-1 model, which was fine-tuned on over 150,000 unlabeled posts from Reachout.com. Our top system had a macroaveraged F1 score of 0.572, providing a new state-of-the-art result on the CLPsych 2017 task. This was achieved without additional information from metadata or preceding posts. Error analyses revealed that this top system often misses expressions of hopelessness. In addition, we have presented visualizations that aid in the understanding of the learned classifiers. CONCLUSIONS In this study, we found that transfer learning is an effective strategy for predicting risk with relatively little labeled data and noted that fine-tuning of pretrained language models provides further gains when large amounts of unlabeled text are available.

scholarly journals Reinforced Logics

1966 ◽  
Vol 28 ◽  
pp. 15-25 ◽  
Author(s):  
Katuzi Ono
Keyword(s):  
Higher Order ◽  
Universal Sentence

The device of representing a predicate by a universal sentence gives rise to logics of higher order. To represent an object constructed by a series of certain steps, we usually use terms. To represent a predicate constructed by a series of certain steps, we now use sentences in general. We can now substitute special predicates represented by sentences for predicate variables of a sentence just as we have been able to substitute special objects represented by terms for object variables. It is remarkable that we can always distinguish predicates from objects, proceeding in this way.

scholarly journals Lattices of pseudovarieties

1989 ◽  
Vol 46 (2) ◽  
pp. 177-183 ◽  
Author(s):  
P. Agliano ◽  
J. B. Nation
Keyword(s):  
Finite Lattice ◽  
Finite Variety ◽  
Locally Finite ◽  
Finite Height ◽  
Lattice Of Subvarieties ◽  
Universal Sentence ◽  
Locally Finite Variety

AbstractWe consider the lattice of pseudovarieties contained in a given pseudovariety P. It is shown that if the lattice L of subpseudovarieties of P has finite height, then L is isomorphic to the lattice of subvarieties of a locally finite variety. Thus not every finite lattice is isomorphic to a lattice of subpseudovarieties. Moreover, the lattice of subpseudovarieties of P satisfies every positive universal sentence holding in all lattice of subvarieties of varieties V(A) ganarated by algebras A ε P.

scholarly journals DLI-IT: A Deep Learning Approach to Drug Label Identification Through Image and Text Embedding

2020 ◽  
Author(s):  
Xiangwen Liu ◽  
Joe Meehan ◽  
Weida Tong ◽  
Leihong Wu ◽  
Xiaowei Xu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Package Insert ◽  
Drug Distribution ◽  
Labor Cost ◽  
Drug Label ◽  
Accurate Identification ◽  
Learning Framework ◽  
Universal Sentence ◽  
Reference Images ◽  
Text Information

Abstract [Background] Drug label, or packaging insert play a significant role in all the operations from production through drug distribution channels to the end consumer. Image of the label also called Display Panel or label could be used to identify illegal, illicit, unapproved and potentially dangerous drugs. Due to the time-consuming process and high labor cost of investigation, an artificial intelligence-based deep learning model is necessary for fast and accurate identification of the drugs. [Methods] In addition to image-based identification technology, we take advantages of rich text information on the pharmaceutical package insert of drug label images. In this study, we developed the Drug Label Identification through Image and Text embedding model (DLI-IT) to model text-based patterns of historical data for detection of suspicious drugs. In DLI-IT, we first trained a Connectionist Text Proposal Network (CTPN) to crop the raw image into sub-images based on the text. The texts from the cropped sub-images are recognized independently through the Tesseract OCR Engine and combined as one document for each raw image. Finally, we applied universal sentence embedding to transform these documents into vectors and find the most similar reference images to the test image through the cosine similarity. [Results] We trained the DLI-IT model on 1749 opioid and 2365 non-opioid drug label images. The model was then tested on 300 external opioid drug label images, the result demonstrated our model achieves up-to 88% of the precision in drug label identification, which outperforms previous image-based or text-based identification method by up-to 35% improvement. [Conclusion] To conclude, by combining Image and Text embedding analysis under deep learning framework, our DLI-IT approach achieved a competitive performance in advancing drug label identification.

scholarly journals Universal Sentence Encoder for English

2018 ◽  
Author(s):  
Daniel Cer ◽  
Yinfei Yang ◽  
Sheng-yi Kong ◽  
Nan Hua ◽  
Nicole Limtiaco ◽  
...  
Keyword(s):  
Universal Sentence

scholarly journals Video Caption Based Searching Using End-to-End Dense Captioning and Sentence Embeddings

Symmetry ◽  
2020 ◽  
Vol 12 (6) ◽  
pp. 992
Author(s):  
Akshay Aggarwal ◽  
Aniruddha Chauhan ◽  
Deepika Kumar ◽  
Mamta Mittal ◽  
Sudipta Roy ◽  
...  
Keyword(s):  
Search Space ◽  
Video Content ◽  
Video Captioning ◽  
The Past ◽  
Percentile Score ◽  
Universal Sentence ◽  
End To End ◽  
Video Searching ◽  
Searching Method

Traditionally, searching for videos on popular streaming sites like YouTube is performed by taking the keywords, titles, and descriptions that are already tagged along with the video into consideration. However, the video content is not utilized for searching of the user’s query because of the difficulty in encoding the events in a video and comparing them to the search query. One solution to tackle this problem is to encode the events in a video and then compare them to the query in the same space. A method of encoding meaning to a video could be video captioning. The captioned events in the video can be compared to the query of the user, and we can get the optimal search space for the videos. There have been many developments over the course of the past few years in modeling video-caption generators and sentence embeddings. In this paper, we exploit an end-to-end video captioning model and various sentence embedding techniques that collectively help in building the proposed video-searching method. The YouCook2 dataset was used for the experimentation. Seven sentence embedding techniques were used, out of which the Universal Sentence Encoder outperformed over all the other six, with a median percentile score of 99.51. Thus, this method of searching, when integrated with traditional methods, can help improve the quality of search results.

Sign in / Sign up

Export Citation Format

Share Document