scholarly journals Data-Augmented Hybrid Named Entity Recognition for Disaster Management by Transfer Learning

2020 ◽  
Vol 10 (12) ◽  
pp. 4234 ◽  
Author(s):  
Hung-Kai Kung ◽  
Chun-Mo Hsieh ◽  
Cheng-Yu Ho ◽  
Yun-Cheng Tsai ◽  
Hao-Yung Chan ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Disaster Management ◽  
Data Augmentation ◽  
Reference Model ◽  
Conditional Random Field ◽  
Hybrid Approach ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Data Set ◽  
Named Entity

This research aims to build a Mandarin named entity recognition (NER) module using transfer learning to facilitate damage information gathering and analysis in disaster management. The hybrid NER approach proposed in this research includes three modules: (1) data augmentation, which constructs a concise data set for disaster management; (2) reference model, which utilizes the bidirectional long short-term memory–conditional random field framework to implement NER; and (3) the augmented model built by integrating the first two modules via cross-domain transfer with disparate label sets. Through the combination of established rules and learned sentence patterns, the hybrid approach performs well in NER tasks for disaster management and recognizes unfamiliar words successfully. This research applied the proposed NER module to disaster management. In the application, we favorably handled the NER tasks of our related work and achieved our desired outcomes. Through proper transfer, the results of this work can be extended to other fields and consequently bring valuable advantages in diverse applications.

scholarly journals Towards reliable named entity recognition in the biomedical domain

2019 ◽  
Author(s):  
John Giorgi ◽  
Gary Bader
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
High Performance ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Maximal Increase ◽  
Named Entity ◽  
Task Learning

Motivation: Automatic biomedical named entity recognition (BioNER) is a key task in biomedical information extraction (IE). For some time, state-of-the-art BioNER has been dominated by machine learning methods, particularly conditional random fields (CRFs), with a recent focus on deep learning. However, recent work has suggested that the high performance of CRFs for BioNER may not generalize to corpora other than the one it was trained on. In our analysis, we find that a popular deep learning-based approach to BioNER, known as bidirectional long short-term memory network-conditional random field (BiLSTM-CRF), is correspondingly poor at generalizing - often dramatically overfitting the corpus it was trained on. To address this, we evaluate three modifications of BiLSTM-CRF for BioNER to alleviate overfitting and improve generalization: improved regularization via variational dropout, transfer learning, and multi-task learning. Results: We measure the effect that each strategy has when training/testing on the same corpus ("in-corpus" performance) and when training on one corpus and evaluating on another ("out-of-corpus" performance), our measure of the models ability to generalize. We found that variational dropout improves out-of-corpus performance by an average of 4.62%, transfer learning by 6.48% and multi-task learning by 8.42%. The maximal increase we identified combines multi-task learning and variational dropout, which boosts out-of-corpus performance by 10.75%. Furthermore, we make available a new open-source tool, called Saber, that implements our best BioNER models. Availability: Source code for our biomedical IE tool is available at https://github.com/BaderLab/saber. Corpora and other resources used in this study are available at https://github.com/BaderLab/Towards- reliable-BioNER.

Named entity recognition of local adverse drug reactions in Xinjiang based on transfer learning

2021 ◽  
Vol 40 (5) ◽  
pp. 8899-8914
Author(s):  
Keming Kang ◽  
Shengwei Tian ◽  
Long Yu
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Adverse Drug Reactions ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Learning Ability ◽  
Data Sets ◽  
Drug Reactions ◽  
Data Set ◽  
Named Entity

For deep learning’s insufficient learning ability of a small amount of data in the Chinese named entity recognition based on deep learning, this paper proposes a named entity recognition of local adverse drug reactions based on Adversarial Transfer Learning, and constructs a neural network model ASAIBC consisting of Adversarial Transfer Learning, Self-Attention, independently recurrent neural network (IndRNN), Bi-directional long short-term memory (BiLSTM) and conditional random field (CRF). However, of the task of Chinese named entity recognition (NER), there are only few open labeled data sets. Therefore, this article introduces Adversarial Transfer Learning network to fully utilize the boundary of Chinese word segmentation tasks (CWS) and NER tasks for information sharing. Plus, the specific information in the CWS is also filtered. Combing with Self-Attention mechanism and IndRNN, this feature’s expression ability is enhanced, thus allowing the model to concern the important information of different entities from different levels. Along with better capture of the dependence relations of long sentences, the recognition ability of the model is further strengthened. As all the results gained from WeiBoNER and MSRA data sets by ASAIBC model are better than traditional algorithms, this paper conducts an experiment on the data set of Xinjiang local named entity recognition of adverse drug reactions (XJADRNER) based on manual labeling, with the accuracy, precision, recall and F-Score value being 98.97%, 91.01%, 90.21% and 90.57% respectively. These experimental results have shown that ASAIBC model can significantly improve the NER performance of local adverse drug reactions in Xinjiang.

A named entity recognition model based on ensemble learning

2020 ◽  
pp. 1-12
Author(s):  
Xinghui Zhu ◽  
Zhuoyang Zou ◽  
Bo Qiao ◽  
Kui Fang ◽  
Yiming Chen
Keyword(s):  
Ensemble Learning ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Knowledge Graph ◽  
Data Set ◽  
Recognition Model ◽  
Named Entity ◽  
Knowledge Graphs

Knowledge Graph has gradually become one of core drivers advancing the Internet and AI in recent years, while there is currently no normal knowledge graph in the field of agriculture. Named Entity Recognition (NER), one important step in constructing knowledge graphs, has become a hot topic in both academia and industry. With the help of the Bidirectional Long Short-Term Memory Network (Bi-LSTM) and Conditional Random Field (CRF) model, we introduce a method of ensemble learning, and implement a named entity recognition model ELER. Our model achieves good results for the CoNLL2003 data set, the accuracy and F1 value in the best experimental results are respectively improved by 1.37% and 0.7% when compared with the BiLSTM-CRF model. In addition, our model achieves an F1 score of 91% for the agricultural data set AgriNER2018, which proves the validity of ELER model for small agriculture sample data sets and lays a foundation for the construction of agricultural knowledge graphs.

scholarly journals Towards reliable named entity recognition in the biomedical domain

2019 ◽  
Vol 36 (1) ◽  
pp. 280-286 ◽  
Author(s):  
John M Giorgi ◽  
Gary D Bader
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Supplementary Information ◽  
Maximal Increase ◽  
Named Entity ◽  
Task Learning

Abstract Motivation Automatic biomedical named entity recognition (BioNER) is a key task in biomedical information extraction. For some time, state-of-the-art BioNER has been dominated by machine learning methods, particularly conditional random fields (CRFs), with a recent focus on deep learning. However, recent work has suggested that the high performance of CRFs for BioNER may not generalize to corpora other than the one it was trained on. In our analysis, we find that a popular deep learning-based approach to BioNER, known as bidirectional long short-term memory network-conditional random field (BiLSTM-CRF), is correspondingly poor at generalizing. To address this, we evaluate three modifications of BiLSTM-CRF for BioNER to improve generalization: improved regularization via variational dropout, transfer learning and multi-task learning. Results We measure the effect that each strategy has when training/testing on the same corpus (‘in-corpus’ performance) and when training on one corpus and evaluating on another (‘out-of-corpus’ performance), our measure of the model’s ability to generalize. We found that variational dropout improves out-of-corpus performance by an average of 4.62%, transfer learning by 6.48% and multi-task learning by 8.42%. The maximal increase we identified combines multi-task learning and variational dropout, which boosts out-of-corpus performance by 10.75%. Furthermore, we make available a new open-source tool, called Saber that implements our best BioNER models. Availability and implementation Source code for our biomedical IE tool is available at https://github.com/BaderLab/saber. Corpora and other resources used in this study are available at https://github.com/BaderLab/Towards-reliable-BioNER. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Transfer learning for Turkish named entity recognition on noisy text

2020 ◽  
Vol 27 (1) ◽  
pp. 35-64
Author(s):  
Emre Kağan Akkaya ◽  
Burcu Can
Keyword(s):  
Transfer Learning ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Noisy Data ◽  
Entity Recognition ◽  
Named Entity ◽  
Latent Features ◽  
Morphologically Rich Languages ◽  
N Gram ◽  
Noisy Text

AbstractIn this article, we investigate using deep neural networks with different word representation techniques for named entity recognition (NER) on Turkish noisy text. We argue that valuable latent features for NER can, in fact, be learned without using any hand-crafted features and/or domain-specific resources such as gazetteers and lexicons. In this regard, we utilize character-level, character n-gram-level, morpheme-level, and orthographic character-level word representations. Since noisy data with NER annotation are scarce for Turkish, we introduce a transfer learning model in order to learn infrequent entity types as an extension to the Bi-LSTM-CRF architecture by incorporating an additional conditional random field (CRF) layer that is trained on a larger (but formal) text and a noisy text simultaneously. This allows us to learn from both formal and informal/noisy text, thus improving the performance of our model further for rarely seen entity types. We experimented on Turkish as a morphologically rich language and English as a relatively morphologically poor language. We obtained an entity-level F1 score of 67.39% on Turkish noisy data and 45.30% on English noisy data, which outperforms the current state-of-art models on noisy text. The English scores are lower compared to Turkish scores because of the intense sparsity in the data introduced by the user writing styles. The results prove that using subword information significantly contributes to learning latent features for morphologically rich languages.

scholarly journals Data Augmentation Techniques on Arabic Data for Named Entity Recognition

2021 ◽  
Vol 189 ◽  
pp. 292-299
Author(s):  
Caroline Sabty ◽  
Islam Omar ◽  
Fady Wasfalla ◽  
Mohamed Islam ◽  
Slim Abdennadher
Keyword(s):  
Data Augmentation ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity ◽  
Augmentation Techniques

scholarly journals Text Mining of Hazard and Operability Analysis Reports Based on Active Learning

Processes ◽  
2021 ◽  
Vol 9 (7) ◽  
pp. 1178
Author(s):  
Zhenhua Wang ◽  
Beike Zhang ◽  
Dong Gao
Keyword(s):  
Active Learning ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Chemical System ◽  
Chemical Safety ◽  
High Quality ◽  
Data Set ◽  
Final Model ◽  
Named Entity ◽  
Sampling Algorithms

In the field of chemical safety, a named entity recognition (NER) model based on deep learning can mine valuable information from hazard and operability analysis (HAZOP) text, which can guide experts to carry out a new round of HAZOP analysis, help practitioners optimize the hidden dangers in the system, and be of great significance to improve the safety of the whole chemical system. However, due to the standardization and professionalism of chemical safety analysis text, it is difficult to improve the performance of traditional models. To solve this problem, in this study, an improved method based on active learning is proposed, and three novel sampling algorithms are designed, Variation of Token Entropy (VTE), HAZOP Confusion Entropy (HCE) and Amplification of Least Confidence (ALC), which improve the ability of the model to understand HAZOP text. In this method, a part of data is used to establish the initial model. The sampling algorithm is then used to select high-quality samples from the data set. Finally, these high-quality samples are used to retrain the whole model to obtain the final model. The experimental results show that the performance of the VTE, HCE, and ALC algorithms are better than that of random sampling algorithms. In addition, compared with other methods, the performance of the traditional model is improved effectively by the method proposed in this paper, which proves that the method is reliable and advanced.

Sign in / Sign up

Export Citation Format

Share Document