scholarly journals Nested Named Entity Recognition via Second-best Sequence Learning and Decoding

2020 ◽  
Vol 8 ◽  
pp. 605-620 ◽  
Author(s):  
Takashi Shibuya ◽  
Eduard Hovy
Keyword(s):  
Random Field ◽  
Sequence Learning ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Neural Model ◽  
Entity Recognition ◽  
Named Entities ◽  
Second Best ◽  
Named Entity ◽  
Better Than

When an entity name contains other names within it, the identification of all combinations of names can become difficult and expensive. We propose a new method to recognize not only outermost named entities but also inner nested ones. We design an objective function for training a neural model that treats the tag sequence for nested entities as the second best path within the span of their parent entity. In addition, we provide the decoding method for inference that extracts entities iteratively from outermost ones to inner ones in an outside-to-inside way. Our method has no additional hyperparameters to the conditional random field based model widely used for flat named entity recognition tasks. Experiments demonstrate that our method performs better than or at least as well as existing methods capable of handling nested entities, achieving F1-scores of 85.82%, 84.34%, and 77.36% on ACE-2004, ACE-2005, and GENIA datasets, respectively.

scholarly journals A Conditional Random Field Approach for Named Entity Recognition in Bengali and Hindi

2009 ◽  
Vol 2 ◽  
Author(s):  
Asif Ekbal ◽  
Sivaji Bandyopadhyay
Keyword(s):  
Random Field ◽  
Conditional Random Field ◽  
Contextual Information ◽  
Named Entity Recognition ◽  
Standard Test ◽  
Entity Recognition ◽  
Indian Languages ◽  
Named Entity ◽  
Test Sets ◽  
Validation Tests

This paper describes the development of Named Entity Recognition (NER) systems for two leading Indian languages, namely Bengali and Hindi, using the Conditional Random Field (CRF) framework. The system makes use of different types of contextual information along with a variety of features that are helpful in predicting the different named entity (NE) classes. This set of features includes language independent as well as language dependent components. We have used the annotated corpora of 122,467 tokens for Bengali and 502,974 tokens for Hindi tagged with a tag set of twelve different NE classes, defined as part of the IJCNLP-08 NER Shared Task for South and South East Asian Languages (SSEAL). We have considered only the tags that denote person names, location names, organization names, number expressions, time expressions and measurement expressions. A number of experiments have been carried out in order to find out the most suitable features for NER in Bengali and Hindi. The system has been tested with the gold standard test sets of 35K for Bengali and 50K tokens for Hindi. Evaluation results in overall f-score values of 81.15% for Bengali and 78.29% for Hindi for the test sets. 10-fold cross validation tests yield f-score values of 83.89% for Bengali and 80.93% for Hindi. ANOVA analysis is performed to show that the performance improvement due to the use of language dependent features is statistically significant.

Sign in / Sign up

Export Citation Format

Share Document