The Impact of Domain-Specific Pre-Training on Named Entity Recognition Tasks in Materials Science

2021 ◽  
Author(s):  
Nicholas Walker ◽  
Amalie Trewartha ◽  
Haoyan Huo ◽  
Sanghoon Lee ◽  
Kevin Cruse ◽  
...  
Keyword(s):  
Materials Science ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity ◽  
Domain Specific ◽  
The Impact

scholarly journals Semi-Supervised Noisy Label Learning for Chinese Medical Named Entity Recognition

2021 ◽  
pp. 1-10
Author(s):  
Zhucong Li ◽  
Zhen Gan ◽  
Baoli Zhang ◽  
Yubo Chen ◽  
Jing Wan ◽  
...  
Keyword(s):  
Hybrid System ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Final Test ◽  
Named Entity ◽  
The Core ◽  
Category Labels ◽  
Adversarial Training ◽  
Core Idea ◽  
The Impact

Abstract This paper describes our approach for the Chinese Medical named entity recognition(MER) task organized by the 2020 China conference on knowledge graph and semantic computing(CCKS) competition. In this task, we need to identify the entity boundary and category labels of six entities from Chinese electronic medical record(EMR). We construct a hybrid system composed of a semi-supervised noisy label learning model based on adversarial training and a rule postprocessing module. The core idea of the hybrid system is to reduce the impact of data noise by optimizing the model results. Besides, we use post-processing rules to correct three cases of redundant labeling, missing labeling, and wrong labeling in the model prediction results. Our method proposed in this paper achieved strict criteria of 0.9156 and relax criteria of 0.9660 on the final test set, ranking first.

scholarly journals A Probability based Classification of Named Entities for Malayalam Language combining Word, Part of Speech and Lexicalized features

2019 ◽  
Vol 8 (2) ◽  
pp. 839-842
Keyword(s):  
Named Entity Recognition ◽  
Entity Recognition ◽  
Supervised Machine Learning ◽  
Named Entities ◽  
Named Entity ◽  
Domain Specific ◽  
Part Of Speech ◽  
Classification Probability ◽  
Malayalam Language

Named Entity Recognition is the process wherein named entities which are designators of a sentence are identified. Designators of a sentence are domain specific. The proposed system identifies named entities in Malayalam language belonging to tourism domain which generally includes names of persons, places, organizations, dates etc. The system uses word, part of speech and lexicalized features to find the probability of a word belonging to a named entity category and to do the appropriate classification. Probability is calculated based on supervised machine learning using word and part of speech features present in a tagged training corpus and using certain rules applied based on lexicalized features.

scholarly journals BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition

2020 ◽  
Author(s):  
Usman Naseem ◽  
Matloob Khushi ◽  
Vinay Reddy ◽  
Sakthivel Rajendran ◽  
Imran Razzak ◽  
...  
Keyword(s):  
State Of The Art ◽  
Language Model ◽  
Named Entity Recognition ◽  
Training Data ◽  
Entity Recognition ◽  
Future Research ◽  
Named Entity ◽  
Domain Specific ◽  
Context Dependent ◽  
Biomedical Named Entity Recognition

Abstract Background: In recent years, with the growing amount of biomedical documents, coupled with advancement in natural language processing algorithms, the research on biomedical named entity recognition (BioNER) has increased exponentially. However, BioNER research is challenging as NER in the biomedical domain are: (i) often restricted due to limited amount of training data, (ii) an entity can refer to multiple types and concepts depending on its context and, (iii) heavy reliance on acronyms that are sub-domain specific. Existing BioNER approaches often neglect these issues and directly adopt the state-of-the-art (SOTA) models trained in general corpora which often yields unsatisfactory results. Results: We propose biomedical ALBERT (A Lite Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) - bioALBERT - an effective domain-specific pre-trained language model trained on huge biomedical corpus designed to capture biomedical context-dependent NER. We adopted self-supervised loss function used in ALBERT that targets on modelling inter-sentence coherence to better learn context-dependent representations and incorporated parameter reduction strategies to minimise memory usage and enhance the training time in BioNER. In our experiments, BioALBERT outperformed comparative SOTA BioNER models on eight biomedical NER benchmark datasets with four different entity types. The performance is increased for; (i) disease type corpora by 7.47% (NCBI-disease) and 10.63% (BC5CDR-disease); (ii) drug-chem type corpora by 4.61% (BC5CDR-Chem) and 3.89 (BC4CHEMD); (iii) gene-protein type corpora by 12.25% (BC2GM) and 6.42% (JNLPBA); and (iv) Species type corpora by 6.19% (LINNAEUS) and 23.71% (Species-800) is observed which leads to a state-of-the-art results. Conclusions: The performance of proposed model on four different biomedical entity types shows that our model is robust and generalizable in recognizing biomedical entities in text. We trained four different variants of BioALBERT models which are available for the research community to be used in future research.

Assessing the Impact of Contextual Embeddings for Portuguese Named Entity Recognition

2019 ◽  
Author(s):  
Joaquim Santos ◽  
Bernardo Consoli ◽  
Cicero dos Santos ◽  
Juliano Terra ◽  
Sandra Collonini ◽  
...  
Keyword(s):  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity ◽  
The Impact
Sign in / Sign up

Export Citation Format

Share Document