scholarly journals Cross-Sentence N-ary Relation Extraction with Graph LSTMs

2017 ◽  
Vol 5 ◽  
pp. 101-115 ◽  
Author(s):  
Nanyun Peng ◽  
Hoifung Poon ◽  
Chris Quirk ◽  
Kristina Toutanova ◽  
Wen-tau Yih
Keyword(s):  
Short Term Memory ◽  
General Relation ◽  
General Setting ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Task Learning ◽  
Distant Supervision ◽  
Sentence Extraction ◽  
Discourse Relations ◽  
The Impact

Past work in relation extraction has focused on binary relations in single sentences. Recent NLP inroads in high-value domains have sparked interest in the more general setting of extracting n-ary relations that span multiple sentences. In this paper, we explore a general relation extraction framework based on graph long short-term memory networks (graph LSTMs) that can be easily extended to cross-sentence n-ary relation extraction. The graph formulation provides a unified way of exploring different LSTM approaches and incorporating various intra-sentential and inter-sentential dependencies, such as sequential, syntactic, and discourse relations. A robust contextual representation is learned for the entities, which serves as input to the relation classifier. This simplifies handling of relations with arbitrary arity, and enables multi-task learning with related relations. We evaluate this framework in two important precision medicine settings, demonstrating its effectiveness with both conventional supervised learning and distant supervision. Cross-sentence extraction produced larger knowledge bases. and multi-task learning significantly improved extraction accuracy. A thorough analysis of various LSTM approaches yielded useful insight the impact of linguistic analysis on extraction accuracy.

scholarly journals Improving Distantly Supervised Relation Extraction with Neural Noise Converter and Conditional Optimal Selector

2019 ◽  
Vol 33 ◽  
pp. 7273-7280 ◽  
Author(s):  
Shanchan Wu ◽  
Kai Fan ◽  
Qiong Zhang
Keyword(s):  
Transition Matrix ◽  
Noisy Data ◽  
Relation Extraction ◽  
The Other ◽  
Neural Noise ◽  
Distant Supervision ◽  
Other Hand ◽  
The Impact ◽  
Large Corpus

Distant supervised relation extraction has been successfully applied to large corpus with thousands of relations. However, the inevitable wrong labeling problem by distant supervision will hurt the performance of relation extraction. In this paper, we propose a method with neural noise converter to alleviate the impact of noisy data, and a conditional optimal selector to make proper prediction. Our noise converter learns the structured transition matrix on logit level and captures the property of distant supervised relation extraction dataset. The conditional optimal selector on the other hand helps to make proper prediction decision of an entity pair even if the group of sentences is overwhelmed by no-relation sentences. We conduct experiments on a widely used dataset and the results show significant improvement over competitive baseline methods.

scholarly journals Learning to Transfer Relational Representations through Analogy

2019 ◽  
Vol 33 ◽  
pp. 10015-10016
Author(s):  
Gaetano Rossiello ◽  
Alfio Gliozzo ◽  
Michael Glass
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
The State ◽  
Large Set ◽  
Relational Information ◽  
Siamese Network ◽  
Distant Supervision ◽  
Novel Approach ◽  
Art Methods

We propose a novel approach to learn representations of relations expressed by their textual mentions. In our assumption, if two pairs of entities belong to the same relation, then those two pairs are analogous. We collect a large set of analogous pairs by matching triples in knowledge bases with web-scale corpora through distant supervision. This dataset is adopted to train a hierarchical siamese network in order to learn entity-entity embeddings which encode relational information through the different linguistic paraphrasing expressing the same relation. The model can be used to generate pre-trained embeddings which provide a valuable signal when integrated into an existing neural-based model by outperforming the state-of-the-art methods on a relation extraction task.

scholarly journals Biomedical Relation Extraction Using Distant Supervision

2020 ◽  
Vol 2020 ◽  
pp. 1-9 ◽  
Author(s):  
Nada Boudjellal ◽  
Huaping Zhang ◽  
Asif Khan ◽  
Arshad Ahmad
Keyword(s):  
Big Data ◽  
Information Extraction ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Structured Data ◽  
Distant Supervision ◽  
Future Challenges ◽  
Unstructured Information ◽  
Biomedical Relation Extraction

With the accelerating growth of big data, especially in the healthcare area, information extraction is more needed currently than ever, for it can convey unstructured information into an easily interpretable structured data. Relation extraction is the second of the two important tasks of relation extraction. This study presents an overview of relation extraction using distant supervision, providing a generalized architecture of this task based on the state-of-the-art work that proposed this method. Besides, it surveys the methods used in the literature targeting this topic with a description of different knowledge bases used in the process along with the corpora, which can be helpful for beginner practitioners seeking knowledge on this subject. Moreover, the limitations of the proposed approaches and future challenges were highlighted, and possible solutions were proposed.

scholarly journals Cross-Relation Cross-Bag Attention for Distantly-Supervised Relation Extraction

2019 ◽  
Vol 33 ◽  
pp. 419-426 ◽  
Author(s):  
Yujin Yuan ◽  
Liyuan Liu ◽  
Siliang Tang ◽  
Zhongfei Zhang ◽  
Yueting Zhuang ◽  
...  
Keyword(s):  
Selective Attention ◽  
Supervised Learning ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Training Data ◽  
Distant Supervision ◽  
Sentence Level ◽  
Noise Robust

Distant supervision leverages knowledge bases to automatically label instances, thus allowing us to train relation extractor without human annotations. However, the generated training data typically contain massive noise, and may result in poor performances with the vanilla supervised learning. In this paper, we propose to conduct multi-instance learning with a novel Cross-relation Cross-bag Selective Attention (C2SA), which leads to noise-robust training for distant supervised relation extractor. Specifically, we employ the sentence-level selective attention to reduce the effect of noisy or mismatched sentences, while the correlation among relations were captured to improve the quality of attention weights. Moreover, instead of treating all entity-pairs equally, we try to pay more attention to entity-pairs with a higher quality. Similarly, we adopt the selective attention mechanism to achieve this goal. Experiments with two types of relation extractor demonstrate the superiority of the proposed approach over the state-of-the-art, while further ablation studies verify our intuitions and demonstrate the effectiveness of our proposed two techniques.

A Customized Attention-Based Long Short-Term Memory Network for Distant Supervised Relation Extraction

2017 ◽  
Vol 29 (7) ◽  
pp. 1964-1985 ◽  
Author(s):  
Dengchao He ◽  
Hongjun Zhang ◽  
Wenning Hao ◽  
Rui Zhang ◽  
Kai Cheng
Keyword(s):  
False Positive ◽  
Short Term Memory ◽  
Relation Extraction ◽  
Short Term ◽  
Training Corpus ◽  
Term Memory ◽  
Distant Supervision ◽  
Positive Data ◽  
Memory Network ◽  
Long Short Term Memory

Distant supervision, a widely applied approach in the field of relation extraction can automatically generate large amounts of labeled training corpus with minimal manual effort. However, the labeled training corpus may have many false-positive data, which would hurt the performance of relation extraction. Moreover, in traditional feature-based distant supervised approaches, extraction models adopt human design features with natural language processing. It may also cause poor performance. To address these two shortcomings, we propose a customized attention-based long short-term memory network. Our approach adopts word-level attention to achieve better data representation for relation extraction without manually designed features to perform distant supervision instead of fully supervised relation extraction, and it utilizes instance-level attention to tackle the problem of false-positive data. Experimental results demonstrate that our proposed approach is effective and achieves better performance than traditional methods.

scholarly journals Extraction of chemical–protein interactions from the literature using neural networks and narrow instance representation

Database ◽  
2019 ◽  
Vol 2019 ◽  
Author(s):  
Rui Antunes ◽  
Sérgio Matos
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Protein Interactions ◽  
Short Term Memory ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Individual Variability ◽  
Biomedical Literature ◽  
Complex Sentence ◽  
The Individual

Abstract The scientific literature contains large amounts of information on genes, proteins, chemicals and their interactions. Extraction and integration of this information in curated knowledge bases help researchers support their experimental results, leading to new hypotheses and discoveries. This is especially relevant for precision medicine, which aims to understand the individual variability across patient groups in order to select the most appropriate treatments. Methods for improved retrieval and automatic relation extraction from biomedical literature are therefore required for collecting structured information from the growing number of published works. In this paper, we follow a deep learning approach for extracting mentions of chemical–protein interactions from biomedical articles, based on various enhancements over our participation in the BioCreative VI CHEMPROT task. A significant aspect of our best method is the use of a simple deep learning model together with a very narrow representation of the relation instances, using only up to 10 words from the shortest dependency path and the respective dependency edges. Bidirectional long short-term memory recurrent networks or convolutional neural networks are used to build the deep learning models. We report the results of several experiments and show that our best model is competitive with more complex sentence representations or network structures, achieving an F1-score of 0.6306 on the test set. The source code of our work, along with detailed statistics, is publicly available.

scholarly journals Exploring Encoder-Decoder Model for Distant Supervised Relation Extraction

2018 ◽  
Author(s):  
Sen Su ◽  
Ningning Jia ◽  
Xiang Cheng ◽  
Shuguang Zhu ◽  
Ruiping Li
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Relation Extraction ◽  
Short Term ◽  
Sequential Prediction ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Model Training ◽  
The Impact ◽  
Model Relation

In this paper, we present an encoder-decoder model for distant supervised relation extraction. Given an entity pair and its sentence bag as input, in the encoder component, we employ the convolutional neural network to extract the features of the sentences in the sentence bag and merge them into a bag representation. In the decoder component, we utilize the long short-term memory network to model relation dependencies and predict the target relations in a sequential manner. In particular, to enable the sequential prediction of relations, we introduce a measure to quantify the amounts of information the relations take in their sentence bag, and use such information to determine the order of the relations of a sentence bag during model training. Moreover, we incorporate the attention mechanism into our model to dynamically adjust the bag representation to reduce the impact of sentences whose corresponding relations have been predicted. Extensive experiments on a popular dataset show that our model achieves significant improvement over state-of-the-art methods.

scholarly journals Bias Modeling for Distantly Supervised Relation Extraction

2015 ◽  
Vol 2015 ◽  
pp. 1-10 ◽  
Author(s):  
Yang Xiang ◽  
Yaoyun Zhang ◽  
Xiaolong Wang ◽  
Yang Qin ◽  
Wenying Han
Keyword(s):  
Language Processing ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Training Data ◽  
Free Text ◽  
Distant Supervision ◽  
Annotation Process ◽  
Noise Tolerant

Distant supervision (DS) automatically annotates free text with relation mentions from existing knowledge bases (KBs), providing a way to alleviate the problem of insufficient training data for relation extraction in natural language processing (NLP). However, the heuristic annotation process does not guarantee the correctness of the generated labels, promoting a hot research issue on how to efficiently make use of the noisy training data. In this paper, we model two types of biases to reduce noise: (1)bias-distto model the relative distance between points (instances) and classes (relation centers); (2)bias-rewardto model the possibility of each heuristically generated label being incorrect. Based on the biases, we propose three noise tolerant models:MIML-dist,MIML-dist-classify, andMIML-reward, building on top of a state-of-the-art distantly supervised learning algorithm. Experimental evaluations compared with three landmark methods on the KBP dataset validate the effectiveness of the proposed methods.

scholarly journals A hybrid approach toward biomedical relation extraction training corpora: combining distant supervision with crowdsourcing

Database ◽  
2020 ◽  
Vol 2020 ◽  
Author(s):  
Diana Sousa ◽  
Andre Lamurias ◽  
Francisco M Couto
Keyword(s):  
Hybrid Approach ◽  
Relation Extraction ◽  
Knowledge Bases ◽  
Amazon Mechanical Turk ◽  
Domain Expert ◽  
Human Phenotype ◽  
Distant Supervision ◽  
Original Dataset ◽  
Partial Domain ◽  
Biomedical Relation Extraction

Abstract Biomedical relation extraction (RE) datasets are vital in the construction of knowledge bases and to potentiate the discovery of new interactions. There are several ways to create biomedical RE datasets, some more reliable than others, such as resorting to domain expert annotations. However, the emerging use of crowdsourcing platforms, such as Amazon Mechanical Turk (MTurk), can potentially reduce the cost of RE dataset construction, even if the same level of quality cannot be guaranteed. There is a lack of power of the researcher to control who, how and in what context workers engage in crowdsourcing platforms. Hence, allying distant supervision with crowdsourcing can be a more reliable alternative. The crowdsourcing workers would be asked only to rectify or discard already existing annotations, which would make the process less dependent on their ability to interpret complex biomedical sentences. In this work, we use a previously created distantly supervised human phenotype–gene relations (PGR) dataset to perform crowdsourcing validation. We divided the original dataset into two annotation tasks: Task 1, 70% of the dataset annotated by one worker, and Task 2, 30% of the dataset annotated by seven workers. Also, for Task 2, we added an extra rater on-site and a domain expert to further assess the crowdsourcing validation quality. Here, we describe a detailed pipeline for RE crowdsourcing validation, creating a new release of the PGR dataset with partial domain expert revision, and assess the quality of the MTurk platform. We applied the new dataset to two state-of-the-art deep learning systems (BiOnt and BioBERT) and compared its performance with the original PGR dataset, as well as combinations between the two, achieving a 0.3494 increase in average F-measure. The code supporting our work and the new release of the PGR dataset is available at https://github.com/lasigeBioTM/PGR-crowd.

Distant Supervision for Relation Extraction with Matrix Completion

2014 ◽  
Author(s):  
Miao Fan ◽  
Deli Zhao ◽  
Qiang Zhou ◽  
Zhiyuan Liu ◽  
Thomas Fang Zheng ◽  
...  
Keyword(s):  
Matrix Completion ◽  
Relation Extraction ◽  
Distant Supervision
Sign in / Sign up

Export Citation Format

Share Document