scholarly journals Binary Classifiers and Latent Sequence Models for Emotion Detection in Suicide Notes

2012 ◽  
Vol 5s1 ◽  
pp. BII.S8933 ◽  
Author(s):  
Colin Cherry ◽  
Saif M. Mohammad ◽  
Berry De Bruijn
Keyword(s):  
National Research Council ◽  
Training Data ◽  
Emotion Detection ◽  
Suicide Notes ◽  
Comparable Performance ◽  
Fast Development ◽  
Suicide Note ◽  
Sentence Classification ◽  
Binary Classifiers ◽  
F Measure

This paper describes the National Research Council of Canada's submission to the 2011 i2b2 NLP challenge on the detection of emotions in suicide notes. In this task, each sentence of a suicide note is annotated with zero or more emotions, making it a multi-label sentence classification task. We employ two distinct large-margin models capable of handling multiple labels. The first uses one classifier per emotion, and is built to simplify label balance issues and to allow extremely fast development. This approach is very effective, scoring an F-measure of 55.22 and placing fourth in the competition, making it the best system that does not use web-derived statistics or re-annotated training data. Second, we present a latent sequence model, which learns to segment the sentence into a number of emotion regions. This model is intended to gracefully handle sentences that convey multiple thoughts and emotions. Preliminary work with the latent sequence model shows promise, resulting in comparable performance using fewer features.

scholarly journals Three Hybrid Classifiers for the Detection of Emotions in Suicide Notes

2012 ◽  
Vol 5s1 ◽  
pp. BII.S8967 ◽  
Author(s):  
Maria Liakata ◽  
Jee-Hyub Kim ◽  
Shyamasree Saha ◽  
Janna Hastings ◽  
Dietrich Rebholz-Schuhmann
Keyword(s):  
Imbalanced Data ◽  
Training Data ◽  
Annotation Scheme ◽  
The Third ◽  
Hybrid Approaches ◽  
Suicide Notes ◽  
Binary Classifiers ◽  
Hybrid Classifiers ◽  
Multi Class Classification ◽  
Derived Rules

We describe our approach for creating a system able to detect emotions in suicide notes. Motivated by the sparse and imbalanced data as well as the complex annotation scheme, we have considered three hybrid approaches for distinguishing between the different categories. Each of the three approaches combines machine learning with manually derived rules, where the latter target very sparse emotion categories. The first approach considers the task as single label multi-class classification, where an SVM and a CRF classifier are trained to recognise fifteen different categories and their results are combined. Our second approach trains individual binary classifiers (SVM and CRF) for each of the fifteen sentence categories and returns the union of the classifiers as the final result. Finally, our third approach is a combination of binary and multi-class classifiers (SVM and CRF) trained on different subsets of the training data. We considered a number of different feature configurations. All three systems were tested on 300 unseen messages. Our second system had the best performance of the three, yielding an F1 score of 45.6% and a Precision of 60.1% whereas our best Recall (43.6%) was obtained using the third system.

scholarly journals Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction

2020 ◽  
Vol 10 (17) ◽  
pp. 5758
Author(s):  
Injy Sarhan ◽  
Marco Spruit
Keyword(s):  
Information Extraction ◽  
Transfer Learning ◽  
Language Processing ◽  
Relation Extraction ◽  
Training Data ◽  
Open Information Extraction ◽  
Comparable Performance ◽  
Medical Domain ◽  
Inductive Transfer Learning ◽  
F Measure

Various tasks in natural language processing (NLP) suffer from lack of labelled training data, which deep neural networks are hungry for. In this paper, we relied upon features learned to generate relation triples from the open information extraction (OIE) task. First, we studied how transferable these features are from one OIE domain to another, such as from a news domain to a bio-medical domain. Second, we analyzed their transferability to a semantically related NLP task, namely, relation extraction (RE). We thereby contribute to answering the question: can OIE help us achieve adequate NLP performance without labelled data? Our results showed comparable performance when using inductive transfer learning in both experiments by relying on a very small amount of the target data, wherein promising results were achieved. When transferring to the OIE bio-medical domain, we achieved an F-measure of 78.0%, only 1% lower when compared to traditional learning. Additionally, transferring to RE using an inductive approach scored an F-measure of 67.2%, which was 3.8% lower than training and testing on the same task. Hereby, our analysis shows that OIE can act as a reliable source task.

scholarly journals Suicide Note Sentiment Classification: A Supervised Approach Augmented by Web Data

2012 ◽  
Vol 5s1 ◽  
pp. BII.S8956 ◽  
Author(s):  
Yan Xu ◽  
Yue Wang ◽  
Jiahua Liu ◽  
Zhuowen Tu ◽  
Jian-Tao Sun ◽  
...  
Keyword(s):  
Feature Selection ◽  
Sentiment Classification ◽  
Training Data ◽  
Support Vector ◽  
Web Data ◽  
Fine Grained ◽  
Shallow Parsing ◽  
Suicide Note ◽  
N Gram ◽  
F Measure

Objective To create a sentiment classification system for the Fifth i2b2/VA Challenge Track 2, which can identify thirteen subjective categories and two objective categories. Design We developed a hybrid system using Support Vector Machine (SVM) classifiers with augmented training data from the Internet. Our system consists of three types of classification-based systems: the first system uses spanning n-gram features for subjective categories, the second one uses bag-of-n-gram features for objective categories, and the third one uses pattern matching for infrequent or subtle emotion categories. The spanning n-gram features are selected by a feature selection algorithm that leverages emotional corpus from weblogs. Special normalization of objective sentences is generalized with shallow parsing and external web knowledge. We utilize three sources of web data: the weblog of LiveJournal which helps to improve the feature selection, the eBay List which assists in special normalization of information and instructions categories, and the suicide project web which provides unlabeled data with similar properties as suicide notes. Measurements The performance is evaluated by the overall micro-averaged precision, recall and F-measure. Result Our system achieved an overall micro-averaged F-measure of 0.59. Happiness_peacefulness had the highest F-measure of 0.81. We were ranked as the second best out of 26 competing teams. Conclusion Our results indicated that classifying fine-grained sentiments at sentence level is a non-trivial task. It is effective to divide categories into different groups according to their semantic properties. In addition, our system performance benefits from external knowledge extracted from publically available web data of other purposes; performance can be further enhanced when more training data is available.

scholarly journals A Combined Approach to Emotion Detection in Suicide Notes

2012 ◽  
Vol 5s1 ◽  
pp. BII.S8969 ◽  
Author(s):  
Alexander Pak ◽  
Delphine Bernhard ◽  
Patrick Paroubek ◽  
Cyril Grouin
Keyword(s):  
Machine Learning ◽  
The Other ◽  
Combined Approach ◽  
Emotion Detection ◽  
Suicide Notes ◽  
Clinical Records ◽  
Official Evaluation ◽  
F Measure

In this paper, we present the system we have developed for participating in the second task of the i2b2/VA 2011 challenge dedicated to emotion detection in clinical records. On the official evaluation, we ranked 6th out of 26 participants. Our best configuration, based upon a combination of both a machine-learning based approach and manually-defined transducers, obtained a 0.5383 global F-measure, while the distribution of the other 26 participants’ results is characterized by mean = 0.4875, stdev = 0.0742, min = 0.2967, max = 0.6139, and median = 0.5027. Combination of machine learning and transducer is achieved by computing the union of results from both approaches, each using a hierarchy of sentiment specific classifiers.

scholarly journals A Naïve Bayes Approach to Classifying Topics in Suicide Notes

2012 ◽  
Vol 5s1 ◽  
pp. BII.S8945 ◽  
Author(s):  
Irena Spasić ◽  
Pete Burnap ◽  
Mark Greenwood ◽  
Michael Arribas-Ayllon
Keyword(s):  
Naive Bayes ◽  
Classification Performance ◽  
Naïve Bayes ◽  
Training Data ◽  
Naive Bayes Classifier ◽  
Bayes Classifier ◽  
Naïve Bayes Classifier ◽  
Suicide Notes ◽  
Matching Rules ◽  
F Measure

The authors present a system developed for the 2011 i2b2 Challenge on Sentiment Classification, whose aim was to automatically classify sentences in suicide notes using a scheme of 15 topics, mostly emotions. The system combines machine learning with a rule-based methodology. The features used to represent a problem were based on lexico–semantic properties of individual words in addition to regular expressions used to represent patterns of word usage across different topics. A naïve Bayes classifier was trained using the features extracted from the training data consisting of 600 manually annotated suicide notes. Classification was then performed using the naïve Bayes classifier as well as a set of pattern–matching rules. The classification performance was evaluated against a manually prepared gold standard consisting of 300 suicide notes, in which 1,091 out of a total of 2,037 sentences were associated with a total of 1,272 annotations. The competing systems were ranked using the micro-averaged F-measure as the primary evaluation metric. Our system achieved the F-measure of 53% (with 55% precision and 52% recall), which was significantly better than the average performance of 48.75% achieved by the 26 participating teams.

scholarly journals Emotion Detection in Suicide Notes using Maximum Entropy Classification

2012 ◽  
Vol 5s1 ◽  
pp. BII.S8972 ◽  
Author(s):  
Richard Wicentowski ◽  
Matthew R. Sydes
Keyword(s):  
Maximum Entropy ◽  
Test Data ◽  
Training Data ◽  
Emotion Detection ◽  
Shared Task ◽  
Training Set ◽  
Suicide Notes ◽  
Syntactic Features ◽  
Formed Part

An ensemble of supervised maximum entropy classifiers can accurately detect and identify sentiments expressed in suicide notes. Using lexical and syntactic features extracted from a training set of externally annotated suicide notes, we trained separate classifiers for each of fifteen pre-specified emotions. This formed part of the 2011 i2b2 NLP Shared Task, Track 2. The precision and recall of these classifiers related strongly with the number of occurrences of each emotion in the training data. Evaluating on previously unseen test data, our best system achieved an F1 score of 0.534.

scholarly journals Statistical and Similarity Methods for Classifying Emotion in Suicide Notes

2012 ◽  
Vol 5s1 ◽  
pp. BII.S8958 ◽  
Author(s):  
Kirk Roberts ◽  
Sanda M. Harabagiu
Keyword(s):  
Sentiment Analysis ◽  
Similarity Measures ◽  
Classification Problem ◽  
Shared Task ◽  
Suicide Notes ◽  
F Measure

In this paper we report on the approaches that we developed for the 2011 i2b2 Shared Task on Sentiment Analysis of Suicide Notes. We have cast the problem of detecting emotions in suicide notes as a supervised multi-label classification problem. Our classifiers use a variety of features based on (a) lexical indicators, (b) topic scores, and (c) similarity measures. Our best submission has a precision of 0.551, a recall of 0.485, and a F-measure of 0.516.

scholarly journals Multi-Task Self-Supervised Learning for Disfluency Detection

2020 ◽  
Vol 34 (05) ◽  
pp. 9193-9200
Author(s):  
Shaolei Wang ◽  
Wangxiang Che ◽  
Qi Liu ◽  
Pengda Qin ◽  
Ting Liu ◽  
...  
Keyword(s):  
Supervised Learning ◽  
Large Scale ◽  
Experimental Results ◽  
Training Data ◽  
Competitive Performance ◽  
Test Set ◽  
Full Dataset ◽  
Sentence Classification ◽  
Trained Network

Most existing approaches to disfluency detection heavily rely on human-annotated data, which is expensive to obtain in practice. To tackle the training data bottleneck, we investigate methods for combining multiple self-supervised tasks-i.e., supervised tasks where data can be collected without manual labeling. First, we construct large-scale pseudo training data by randomly adding or deleting words from unlabeled news data, and propose two self-supervised pre-training tasks: (i) tagging task to detect the added noisy words. (ii) sentence classification to distinguish original sentences from grammatically-incorrect sentences. We then combine these two tasks to jointly train a network. The pre-trained network is then fine-tuned using human-annotated disfluency detection training data. Experimental results on the commonly used English Switchboard test set show that our approach can achieve competitive performance compared to the previous systems (trained using the full dataset) by using less than 1% (1000 sentences) of the training data. Our method trained on the full dataset significantly outperforms previous methods, reducing the error by 21% on English Switchboard.

scholarly journals HEDGES EXPRESSIONS IN CONFESSIONAL DISCOURSE OF IDA CRADDOCK’S SUICIDE NOTES

Prosodi ◽  
2020 ◽  
Vol 14 (2) ◽  
pp. 73-86
Author(s):  
Rizkya Fajarani Bahar ◽  
Lisetyo Ariyanti
Keyword(s):  
19Th Century ◽  
Cause Of Death ◽  
Freedom Of Speech ◽  
Personal Story ◽  
The Public ◽  
Suicide Notes ◽  
Suicide Note ◽  
Women Rights ◽  
Confessional Discourse

Some people commited suicide tried to express what they felt and left message explaining the causes of why they committed suicide. The suicide note was written by the person who commited suicide as a purpose to give a sign to other people. One of those people was Ida Craddck who was a 19th century American. She advocated freedom of speech and women rights who committed suicide because of inappropriate decision from the judge. Her books were prosecuted by Anthony Comstock as obscene literature. This study was aimed to examine the hedges expressions that maintained the functions of confessional texts which were used by Craddock. The results found that hedges were used on her confessions to support her criticism and wish to the public. Those criticism and wish were confessed by Craddock to aware the public about people’s freedom condition. Her confessions had function to tell her personal story that led her to suicide which could be learnt by other people so that they could have a better life. Finally, hedges were used to express her uncertainty of the truth of what she confessed about her cause of death.

Emotion detection in suicide notes

2013 ◽  
Vol 40 (16) ◽  
pp. 6351-6358 ◽  
Author(s):  
Bart Desmet ◽  
Véronique Hoste
Keyword(s):  
Emotion Detection ◽  
Suicide Notes
Sign in / Sign up

Export Citation Format

Share Document