scholarly journals How to Adapt Deep Learning Models to a New Domain: The Case of Biomedical Relation Extraction

TecnoLógicas ◽  
2019 ◽  
Vol 22 ◽  
pp. 49-62 ◽  
Author(s):  
Jefferson A. Peña-Torres ◽  
Raúl E. Gutiérrez ◽  
Víctor A. Bucheli ◽  
Fabio A. González
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Intelligent Systems ◽  
Domain Adaptation ◽  
Relation Extraction ◽  
Semantic Relations ◽  
Biomedical Domain ◽  
External Resources ◽  
Short Time ◽  
Biomedical Relation Extraction

In this article, we study the relation extraction problem from Natural Language Processing (NLP) implementing a domain adaptation setting without external resources. We trained a Deep Learning (DL) model for Relation Extraction (RE), which extracts semantic relations in the biomedical domain. However, can the model be applied to different domains? The model should be adaptable to automatically extract relationships across different domains using the DL network. Completely training DL models in a short time is impractical because the models should quickly adapt to different datasets in several domains without delay. Therefore, adaptation is crucial for intelligent systems, where changing factors and unanticipated perturbations are common. In this study, we present a detailed analysis of the problem, as well as preliminary experimentation, results, and their evaluation.

LBERT: Lexically aware Transformer-based Bidirectional Encoder Representation model for learning universal bio-entity relations

2020 ◽  
Author(s):  
Neha Warikoo ◽  
Yung-Chun Chang ◽  
Wen-Lian Hsu
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Predictive Analytics ◽  
Relation Extraction ◽  
Data Representation ◽  
Supplementary Information ◽  
Biomedical Domain ◽  
Critical Function ◽  
Representation Model ◽  
Classification Tasks

Abstract Motivation Natural Language Processing techniques are constantly being advanced to accommodate the influx of data as well as to provide exhaustive and structured knowledge dissemination. Within the biomedical domain, relation detection between bio-entities known as the Bio-Entity Relation Extraction (BRE) task has a critical function in knowledge structuring. Although recent advances in deep learning-based biomedical domain embedding have improved BRE predictive analytics, these works are often task selective or use external knowledge-based pre-/post-processing. In addition, deep learning-based models do not account for local syntactic contexts, which have improved data representation in many kernel classifier-based models. In this study, we propose a universal BRE model, i.e. LBERT, which is a Lexically aware Transformer-based Bidirectional Encoder Representation model, and which explores both local and global contexts representations for sentence-level classification tasks. Results This article presents one of the most exhaustive BRE studies ever conducted over five different bio-entity relation types. Our model outperforms state-of-the-art deep learning models in protein–protein interaction (PPI), drug–drug interaction and protein–bio-entity relation classification tasks by 0.02%, 11.2% and 41.4%, respectively. LBERT representations show a statistically significant improvement over BioBERT in detecting true bio-entity relation for large corpora like PPI. Our ablation studies clearly indicate the contribution of the lexical features and distance-adjusted attention in improving prediction performance by learning additional local semantic context along with bi-directionally learned global context. Availability and implementation Github. https://github.com/warikoone/LBERT. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Syntax-based transfer learning for the task of biomedical relation extraction

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Joël Legrand ◽  
Yannick Toussaint ◽  
Chedy Raïssi ◽  
Adrien Coulet
Keyword(s):  
Transfer Learning ◽  
Language Processing ◽  
Domain Adaptation ◽  
Relation Extraction ◽  
Training Data ◽  
Learning Performance ◽  
Promising Alternative ◽  
Syntactic Features ◽  
The Impact ◽  
Biomedical Relation Extraction

Abstract Background Transfer learning aims at enhancing machine learning performance on a problem by reusing labeled data originally designed for a related, but distinct problem. In particular, domain adaptation consists for a specific task, in reusing training data developedfor the same task but a distinct domain. This is particularly relevant to the applications of deep learning in Natural Language Processing, because they usually require large annotated corpora that may not exist for the targeted domain, but exist for side domains. Results In this paper, we experiment with transfer learning for the task of relation extraction from biomedical texts, using the TreeLSTM model. We empirically show the impact of TreeLSTM alone and with domain adaptation by obtaining better performances than the state of the art on two biomedical relation extraction tasks and equal performances for two others, for which little annotated data are available. Furthermore, we propose an analysis of the role that syntactic features may play in transfer learning for relation extraction. Conclusion Given the difficulty to manually annotate corpora in the biomedical domain, the proposed transfer learning method offers a promising alternative to achieve good relation extraction performances for domains associated with scarce resources. Also, our analysis illustrates the importance that syntax plays in transfer learning, underlying the importance in this domain to privilege approaches that embed syntactic features.

scholarly journals Word Embedding for Semantically Relative Words: an Experimental Study

2018 ◽  
Vol 25 (6) ◽  
pp. 726-733
Author(s):  
Maria S. Karyaeva ◽  
Pavel I. Braslavski ◽  
Valery A. Sokolov
Keyword(s):  
Experimental Study ◽  
Natural Language Processing ◽  
Language Processing ◽  
Intelligent Systems ◽  
Russian Language ◽  
Word Embedding ◽  
Semantic Relations ◽  
Automatic Extraction ◽  
Semantic Relationships ◽  
The Russian Language

The ability to identify semantic relations between words has made a word2vec model widely used in NLP tasks. The idea of word2vec is based on a simple rule that a higher similarity can be reached if two words have a similar context. Each word can be represented as a vector, so the closest coordinates of vectors can be interpreted as similar words. It allows to establish semantic relations (synonymy, relations of hypernymy and hyponymy and other semantic relations) by applying an automatic extraction. The extraction of semantic relations by hand is considered as a time-consuming and biased task, requiring a large amount of time and some help of experts. Unfortunately, the word2vec model provides an associative list of words which does not consist of relative words only. In this paper, we show some additional criteria that may be applicable to solve this problem. Observations and experiments with well-known characteristics, such as word frequency, a position in an associative list, might be useful for improving results for the task of extraction of semantic relations for the Russian language by using word embedding. In the experiments, the word2vec model trained on the Flibusta and pairs from Wiktionary are used as examples with semantic relationships. Semantically related words are applicable to thesauri, ontologies and intelligent systems for natural language processing.

Using multiple Web resources and inference rules to classify Chinese word semantic relation

2018 ◽  
Vol 46 (2) ◽  
pp. 120-126 ◽  
Author(s):  
Shutian Ma ◽  
Yingyi Zhang ◽  
Chengzhi Zhang
Keyword(s):  
Language Processing ◽  
Data Access ◽  
Semantic Relation ◽  
Semantic Relations ◽  
Primary System ◽  
Chinese Word ◽  
Lexical Resources ◽  
Data Set ◽  
Content Type ◽  
External Resources

Purpose The purpose of this paper is to classify Chinese word semantic relations, which are synonyms, antonyms, hyponyms and meronymys. Design/methodology/approach Basically, four simple methods are applied, ontology-based, dictionary-based, pattern-based and morpho-syntactic method. The authors make good use of search engine to build lexical and semantic resources for dictionary-based and pattern-based methods. To improve classification performance with more external resources, they also classify the given word pairs in Chinese and in English at the same time by using machine translation. Findings Experimental results show that the approach achieved an average F1 score of 50.87 per cent, an average accuracy of 70.36 per cent and an average recall of 40.05 per cent over all classification tasks. Synonym and antonym classification achieved high accuracy, i.e. above 90 per cent. Moreover, dictionary-based and pattern-based approaches work effectively on final data set. Originality/value For many natural language processing (NLP) tasks, the step of distinguishing word semantic relation can help to improve system performance, such as information extraction and knowledge graph generation. Currently, common methods for this task rely on large corpora for training or dictionaries and thesauri for inference, where limitation lies in freely data access and keeping built lexical resources up-date. This paper builds a primary system for classifying Chinese word semantic relations by seeking new ways to obtain the external resources efficiently.

scholarly journals Towards relation extraction from Arabic text: a review

2019 ◽  
Vol 5 (5) ◽  
pp. 212-215
Author(s):  
Abeer AlArfaj
Keyword(s):  
Language Processing ◽  
Question Answering ◽  
State Of The Art ◽  
Relation Extraction ◽  
Arabic Language ◽  
Extraction Methods ◽  
The State ◽  
Semantic Relations ◽  
Arabic Text ◽  
Taxonomic Relation

Semantic relation extraction is an important component of ontologies that can support many applications e.g. text mining, question answering, and information extraction. However, extracting semantic relations between concepts is not trivial and one of the main challenges in Natural Language Processing (NLP) Field. The Arabic language has complex morphological, grammatical, and semantic aspects since it is a highly inflectional and derivational language, which makes task even more challenging. In this paper, we present a review of the state of the art for relation extraction from texts, addressing the progress and difficulties in this field. We discuss several aspects related to this task, considering the taxonomic and non-taxonomic relation extraction methods. Majority of relation extraction approaches implement a combination of statistical and linguistic techniques to extract semantic relations from text. We also give special attention to the state of the work on relation extraction from Arabic texts, which need further progress.

scholarly journals Using distant supervision to augment manually annotated data for relation extraction

2019 ◽  
Author(s):  
Peng Su ◽  
Gang Li ◽  
Cathy Wu ◽  
K. Vijay-Shanker
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Relation Extraction ◽  
Biomedical Literature ◽  
Training Data ◽  
Distant Supervision ◽  
Large Size ◽  
Domain Expertise

AbstractSignificant progress has been made in applying deep learning on natural language processing tasks recently. However, deep learning models typically require a large amount of annotated training data while often only small labeled datasets are available for many natural language processing tasks in biomedical literature. Building large-size datasets for deep learning is expensive since it involves considerable human effort and usually requires domain expertise in specialized fields. In this work, we consider augmenting manually annotated data with large amounts of data using distant supervision. However, data obtained by distant supervision is often noisy, we first apply some heuristics to remove some of the incorrect annotations. Then using methods inspired from transfer learning, we show that the resulting models outperform models trained on the original manually annotated sets.

Extraction of similar biomedical terms in biomedical literature mining: Examining the effect of the ratio of biomedical domain to general domain data (Preprint)

2021 ◽  
Author(s):  
Ziheng Zhang ◽  
Feng Han ◽  
Hongjian Zhang ◽  
Tomohiro Aoki ◽  
Katsuhiko Ogasawara
Keyword(s):  
Language Processing ◽  
Relation Extraction ◽  
Medical Data ◽  
Biomedical Literature ◽  
Literature Mining ◽  
Biomedical Domain ◽  
Pubmed Central ◽  
General Domain ◽  
Biomedical Information Retrieval ◽  
Science Community

BACKGROUND Biomedical terms extracted using Word2vec, the most popular word embedding model in recent years, serve as the foundation for various natural language processing (NLP) applications, such as biomedical information retrieval, relation extraction, and recommendation systems. OBJECTIVE The objective of this study is to examine how changes in the ratio of biomedical domain to general domain data in the corpus affect the extraction of similar biomedical terms using Word2vec. METHODS We downloaded abstracts of 214892 articles from PubMed Central (PMC) and the 3.9 GB Billion Word (BW) benchmark corpus from the computer science community. The datasets were preprocessed and grouped into 11 corpora based on the ratio of BW to PMC, ranging from 0:10 to 10:0, and then Word2vec models were trained on these corpora. The cosine similarities between the biomedical terms obtained from the Word2vec models were then compared in each model. RESULTS The results indicated that the models trained with both BW and PMC data outperformed the model trained only with medical data. The similarity between the biomedical terms extracted by the Word2vec model increased, when the ratio of biomedical domain to general domain data was 3: 7 to 5: 5. CONCLUSIONS This study allows NLP researchers to apply Word2vec based on more information and increase the similarity of extracted biomedical terms to improve their effectiveness in NLP applications, such as biomedical information extraction.

scholarly journals BioRel: towards large-scale biomedical relation extraction

2020 ◽  
Vol 21 (S16) ◽  
Author(s):  
Rui Xing ◽  
Jie Luo ◽  
Tengwei Song
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Critical Role ◽  
Relation Extraction ◽  
Extraction Methods ◽  
Statistical Machine Learning ◽  
Language System ◽  
Unified Medical Language System ◽  
Medical Language ◽  
Biomedical Relation Extraction

Abstract Background Although biomedical publications and literature are growing rapidly, there still lacks structured knowledge that can be easily processed by computer programs. In order to extract such knowledge from plain text and transform them into structural form, the relation extraction problem becomes an important issue. Datasets play a critical role in the development of relation extraction methods. However, existing relation extraction datasets in biomedical domain are mainly human-annotated, whose scales are usually limited due to their labor-intensive and time-consuming nature. Results We construct BioRel, a large-scale dataset for biomedical relation extraction problem, by using Unified Medical Language System as knowledge base and Medline as corpus. We first identify mentions of entities in sentences of Medline and link them to Unified Medical Language System with Metamap. Then, we assign each sentence a relation label by using distant supervision. Finally, we adapt the state-of-the-art deep learning and statistical machine learning methods as baseline models and conduct comprehensive experiments on the BioRel dataset. Conclusions Based on the extensive experimental results, we have shown that BioRel is a suitable large-scale datasets for biomedical relation extraction, which provides both reasonable baseline performance and many remaining challenges for both deep learning and statistical methods.

scholarly journals The upsurge of deep learning for computer vision applications

2020 ◽  
Vol 10 (1) ◽  
pp. 538
Author(s):  
Priyanka Patel ◽  
Amit Thakkar
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Cyber Security ◽  
Intelligent Systems ◽  
Communication Process ◽  
Signal Process ◽  
Linguistic Communication ◽  
The Everyday ◽  
Processing Signal ◽  
Computer Vision Applications

Artificial intelligence (AI) is additionally serving to a brand new breed of corporations disrupt industries from restorative examination to horticulture. Computers can’t nevertheless replace humans, however, they will work superbly taking care of the everyday tangle of our lives. The era is reconstructing big business and has been on the rise in recent years which has grounded with the success of deep learning (DL). Cyber-security, Auto and health industry are three industries innovating with AI and DL technologies and also Banking, retail, finance, robotics, manufacturing. The healthcare industry is one of the earliest adopters of AI and DL. DL accomplishing exceptional dimensions levels of accurateness to the point where DL algorithms can outperform humans at classifying videos & images. The major drivers that caused the breakthrough of deep neural networks are the provision of giant amounts of coaching information, powerful machine infrastructure, and advances in academia. DL is heavily employed in each academe to review intelligence and within the trade-in building intelligent systems to help humans in varied tasks. Thereby DL systems begin to crush not solely classical ways, but additionally, human benchmarks in numerous tasks like image classification, action detection, natural language processing, signal process, and linguistic communication process.

Sign in / Sign up

Export Citation Format

Share Document