Text segmentation for patent claim simplification via Bidirectional Long‐Short Term Memory and Conditional Random Field

Pengenalan entitas bernama (named-entity recognition atau NER) adalah proses otomatis mengekstraksi entitas bernama yang dianggap penting di dalam sebuah teks dan menentukan kategorinya ke dalam kategori terdefinisi. Sebagai contoh, untuk teks berita, NER dapat mengekstraksi nama orang, nama organisasi, dan nama lokasi. NER bermanfaat dalam berbagai aplikasi analisis teks, misalnya pencarian, sistem tanya jawab, peringkasan teks dan mesin penerjemah. Tantangan utama NER adalah penanganan ambiguitas makna karena konteks kata pada kalimat, misalnya kata “Cendana” dapat merupakan nama lokasi (Jalan Cendana), atau nama organisasi (Keluarga Cendana), atau nama tanaman. Tantangan lainnya adalah penentuan batas entitas, misalnya “[Istora Senayan] [Jakarta]”. Berbagai kakas NER telah dikembangkan untuk berbagai bahasa terutama Bahasa Inggris dengan kinerja yang baik, tetapi kakas NER bahasa Indonesia masih memiliki kinerja yang belum baik. Makalah ini membahas pendekatan berbasis pembelajaran mesin untuk menghasilkan model NER bahasa Indonesia. Pendekatan ini sangat bergantung pada korpus yang menjadi sumber belajar, dan teknik pembelajaran mesin yang digunakan. Teknik yang akan digunakan adalah LSTM - CRF (Long Short Term Memory – Conditional Random Field). Hasil terbaik (F-measure = 0.72) didapatkan dengan menggunakan word embedding GloVe Wikipedia Bahasa Indonesia.

Download Full-text

An Approach to NMT Re-Ranking Using Sequence-Labeling for Grammatical Error Correction

Journal of Advanced Computational Intelligence and Intelligent Informatics ◽

10.20965/jaciii.2020.p0557 ◽

2020 ◽

Vol 24 (4) ◽

pp. 557-567

Author(s):

Bo Wang ◽

◽

Kaoru Hirota ◽

Chang Liu ◽

Yaping Dai ◽

...

Keyword(s):

Error Correction ◽

Short Term Memory ◽

Conditional Random Field ◽

Short Term ◽

Neural Machine Translation ◽

Term Memory ◽

Sequence Labeling ◽

Grammatical Error ◽

Small Set ◽

Long Short Term Memory

An approach to N-best hypotheses re-ranking using a sequence-labeling model is applied to resolve the data deficiency problem in Grammatical Error Correction (GEC). Multiple candidate sentences are generated using a Neural Machine Translation (NMT) model; thereafter, these sentences are re-ranked via a stacked Transformer following a Bidirectional Long Short-Term Memory (BiLSTM) with Conditional Random Field (CRF). Correlations within the sentences are extracted using the sequence-labeling model based on the Transformer, which is particularly suitable for long sentences. Meanwhile, the knowledge from a large amount of unlabeled data is acquired through the pre-trained structure. Thus, completely revised sentences are adopted instead of partially modified sentences. Compared with conventional NMT, experiments on the NUCLE and FCE datasets demonstrate that the model improves the F0.5 score by 8.22% and 2.09%, respectively. As an advantage, the proposed re-ranking method has the advantage of only requires a small set of easily computed features that do not need linguistic inputs.

Download Full-text

Post Text Processing of Chinese Speech Recognition Based on Bidirectional LSTM Networks and CRF

Electronics ◽

10.3390/electronics8111248 ◽

2019 ◽

Vol 8 (11) ◽

pp. 1248 ◽

Cited By ~ 3

Author(s):

Li Yang ◽

Ying Li ◽

Jin Wang ◽

Zhuo Tang

Keyword(s):

Speech Recognition ◽

Error Detection ◽

Short Term Memory ◽

Text Processing ◽

Conditional Random Field ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory ◽

Lstm Network ◽

Two Stages

With the rapid development of Internet of Things Technology, speech recognition has been applied more and more widely. Chinese Speech Recognition is a complex process. In the process of speech-to-text conversion, due to the influence of dialect, environmental noise, and context, the accuracy of speech-to-text in multi-round dialogues and specific contexts is still not high. After the general speech recognition technology, the text after speech recognition can be detected and corrected in the specific context, which is helpful to improve the robustness of text comprehension and is a beneficial supplement to the speech recognition technology. In this paper, a text processing model after Chinese Speech Recognition is proposed, which combines a bidirectional long short-term memory (LSTM) network with a conditional random field (CRF) model. The task is divided into two stages: text error detection and text error correction. In this paper, a bidirectional long short-term memory (Bi-LSTM) network and conditional random field are used in two stages of text error detection and text error correction respectively. Through verification and system test on the SIGHAN 2013 Chinese Spelling Check (CSC) dataset, the experimental results show that the model can effectively improve the accuracy of text after speech recognition.

Download Full-text

Extracting entities with attributes in clinical text via joint deep learning

Journal of the American Medical Informatics Association ◽

10.1093/jamia/ocz158 ◽

2019 ◽

Vol 26 (12) ◽

pp. 1584-1591 ◽

Cited By ~ 1

Author(s):

Xue Shi ◽

Yingping Yi ◽

Ying Xiong ◽

Buzhou Tang ◽

Qingcai Chen ◽

...

Keyword(s):

Deep Learning ◽

Language Processing ◽

Short Term Memory ◽

Conditional Random Field ◽

Relation Extraction ◽

Entity Recognition ◽

Short Term ◽

Term Memory ◽

Clinical Text ◽

Long Short Term Memory

Abstract Objective Extracting clinical entities and their attributes is a fundamental task of natural language processing (NLP) in the medical domain. This task is typically recognized as 2 sequential subtasks in a pipeline, clinical entity or attribute recognition followed by entity-attribute relation extraction. One problem of pipeline methods is that errors from entity recognition are unavoidably passed to relation extraction. We propose a novel joint deep learning method to recognize clinical entities or attributes and extract entity-attribute relations simultaneously. Materials and Methods The proposed method integrates 2 state-of-the-art methods for named entity recognition and relation extraction, namely bidirectional long short-term memory with conditional random field and bidirectional long short-term memory, into a unified framework. In this method, relation constraints between clinical entities and attributes and weights of the 2 subtasks are also considered simultaneously. We compare the method with other related methods (ie, pipeline methods and other joint deep learning methods) on an existing English corpus from SemEval-2015 and a newly developed Chinese corpus. Results Our proposed method achieves the best F1 of 74.46% on entity recognition and the best F1 of 50.21% on relation extraction on the English corpus, and 89.32% and 88.13% on the Chinese corpora, respectively, which outperform the other methods on both tasks. Conclusions The joint deep learning–based method could improve both entity recognition and relation extraction from clinical text in both English and Chinese, indicating that the approach is promising.

Download Full-text

Attention-Based Joint Entity Linking with Entity Embedding

Information ◽

10.3390/info10020046 ◽

2019 ◽

Vol 10 (2) ◽

pp. 46 ◽

Cited By ~ 4

Author(s):

Chen Liu ◽

Feng Li ◽

Xian Sun ◽

Hongzhe Han

Keyword(s):

Short Term Memory ◽

State Of The Art ◽

Conditional Random Field ◽

Entity Linking ◽

Short Term ◽

High Quality ◽

Term Memory ◽

Entity Disambiguation ◽

Long Short Term Memory ◽

Made In

Entity linking (also called entity disambiguation) aims to map the mentions in a given document to their corresponding entities in a target knowledge base. In order to build a high-quality entity linking system, efforts are made in three parts: Encoding of the entity, encoding of the mention context, and modeling the coherence among mentions. For the encoding of entity, we use long short term memory (LSTM) and a convolutional neural network (CNN) to encode the entity context and entity description, respectively. Then, we design a function to combine all the different entity information aspects, in order to generate unified, dense entity embeddings. For the encoding of mention context, unlike standard attention mechanisms which can only capture important individual words, we introduce a novel, attention mechanism-based LSTM model, which can effectively capture the important text spans around a given mention with a conditional random field (CRF) layer. In addition, we take the coherence among mentions into consideration with a Forward-Backward Algorithm, which is less time-consuming than previous methods. Our experimental results show that our model obtains a competitive, or even better, performance than state-of-the-art models across different datasets.

Download Full-text