scholarly journals Construction of Meteorological Simulation Knowledge Graph Based on Deep Learning Method

2021 ◽  
Vol 13 (3) ◽  
pp. 1311
Author(s):  
Ziwei Xiao ◽  
Chunxiao Zhang
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Research Literature ◽  
Knowledge Graph ◽  
Simulation Research ◽  
Efficient Management ◽  
Structured Knowledge ◽  
Scientific Results ◽  
Meteorological Simulation

With the maturity of meteorological simulation technology, the research literature in this field is undergoing a rapid increase. The published literature can provide useful guidance for current research to get scientific results; however, it tends to be rather time consuming to obtain exact knowledge from massive literature, and it is necessary to transform the literature into structured knowledge to meet the efficient management, sharing, and reuse of meteorological simulation knowledge. In this paper, methods of meteorological simulation knowledge extraction and knowledge graph construction are proposed. A deep learning model based on bilateral long short-term memory-conditional random field (BiLSTM-CRF) is used to extract the meteorological simulation knowledge from the massive literature. Then, the Neo4j graph database is used to construct the meteorological simulation knowledge graph. Based on the meteorological simulation knowledge graph, it can realize the structured storage and integration of meteorological simulation knowledge, which can bridge the gap in the transformation of massive literature to sharable and reusable knowledge. Furthermore, the meteorological simulation knowledge graph can be used as an expert resource and contribute to sustainable guidance and optimization for meteorological simulation research.

Deep Learning Approach for the Morphological Synthesis in Malayalam and Tamil at the Character Level

2021 ◽  
Vol 20 (6) ◽  
pp. 1-17
Author(s):  
B. Premjith ◽  
K. P. Soman
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Short Term ◽  
Long Short Term Memory ◽  
Target Languages ◽  
Main Components ◽  
Learning Architectures ◽  
Gated Recurrent Unit ◽  
Character Sequence

Morphological synthesis is one of the main components of Machine Translation (MT) frameworks, especially when any one or both of the source and target languages are morphologically rich. Morphological synthesis is the process of combining two words or two morphemes according to the Sandhi rules of the morphologically rich language. Malayalam and Tamil are two languages in India which are morphologically abundant as well as agglutinative. Morphological synthesis of a word in these two languages is challenging basically because of the following reasons: (1) Abundance in morphology; (2) Complex Sandhi rules; (3) The possibilty in Malayalam to form words by combining words that belong to different syntactic categories (for example, noun and verb); and (4) The construction of a sentence by combining multiple words. We formulated the task of the morphological generation of nouns and verbs of Malayalam and Tamil as a character-to-character sequence tagging problem. In this article, we used deep learning architectures like Recurrent Neural Network (RNN) , Long Short-Term Memory Networks (LSTM) , Gated Recurrent Unit (GRU) , and their stacked and bidirectional versions for the implementation of morphological synthesis at the character level. In addition to that, we investigated the performance of the combination of the aforementioned deep learning architectures and the Conditional Random Field (CRF) in the morphological synthesis of nouns and verbs in Malayalam and Tamil. We observed that the addition of CRF to the Bidirectional LSTM/GRU architecture achieved more than 99% accuracy in the morphological synthesis of Malayalam and Tamil nouns and verbs.

scholarly journals Deep Learning-Based Named Entity Recognition and Knowledge Graph Construction for Geological Hazards

2019 ◽  
Vol 9 (1) ◽  
pp. 15 ◽  
Author(s):  
Runyu Fan ◽  
Lizhe Wang ◽  
Jining Yan ◽  
Weijing Song ◽  
Yingqian Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Knowledge Graph ◽  
Geological Hazard ◽  
Geological Hazards ◽  
Named Entity ◽  
Corpus Construction

Constructing a knowledge graph of geological hazards literature can facilitate the reuse of geological hazards literature and provide a reference for geological hazard governance. Named entity recognition (NER), as a core technology for constructing a geological hazard knowledge graph, has to face the challenges that named entities in geological hazard literature are diverse in form, ambiguous in semantics, and uncertain in context. This can introduce difficulties in designing practical features during the NER classification. To address the above problem, this paper proposes a deep learning-based NER model; namely, the deep, multi-branch BiGRU-CRF model, which combines a multi-branch bidirectional gated recurrent unit (BiGRU) layer and a conditional random field (CRF) model. In an end-to-end and supervised process, the proposed model automatically learns and transforms features by a multi-branch bidirectional GRU layer and enhances the output with a CRF layer. Besides the deep, multi-branch BiGRU-CRF model, we also proposed a pattern-based corpus construction method to construct the corpus needed for the deep, multi-branch BiGRU-CRF model. Experimental results indicated the proposed deep, multi-branch BiGRU-CRF model outperformed state-of-the-art models. The proposed deep, multi-branch BiGRU-CRF model constructed a large-scale geological hazard literature knowledge graph containing 34,457 entities nodes and 84,561 relations.

scholarly journals Advancing PICO element detection in biomedical text via deep neural networks

2020 ◽  
Vol 36 (12) ◽  
pp. 3856-3862
Author(s):  
Di Jin ◽  
Peter Szolovits
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Contextual Information ◽  
Learning Model ◽  
Detection Accuracy ◽  
Clinical Question ◽  
Specific Patient ◽  
Benchmark Datasets ◽  
Deep Learning Model

Abstract Motivation In evidence-based medicine, defining a clinical question in terms of the specific patient problem aids the physicians to efficiently identify appropriate resources and search for the best available evidence for medical treatment. In order to formulate a well-defined, focused clinical question, a framework called PICO is widely used, which identifies the sentences in a given medical text that belong to the four components typically reported in clinical trials: Participants/Problem (P), Intervention (I), Comparison (C) and Outcome (O). In this work, we propose a novel deep learning model for recognizing PICO elements in biomedical abstracts. Based on the previous state-of-the-art bidirectional long-short-term memory (bi-LSTM) plus conditional random field architecture, we add another layer of bi-LSTM upon the sentence representation vectors so that the contextual information from surrounding sentences can be gathered to help infer the interpretation of the current one. In addition, we propose two methods to further generalize and improve the model: adversarial training and unsupervised pre-training over large corpora. Results We tested our proposed approach over two benchmark datasets. One is the PubMed-PICO dataset, where our best results outperform the previous best by 5.5%, 7.9% and 5.8% for P, I and O elements in terms of F1 score, respectively. And for the other dataset named NICTA-PIBOSO, the improvements for P/I/O elements are 3.9%, 15.6% and 1.3% in F1 score, respectively. Overall, our proposed deep learning model can obtain unprecedented PICO element detection accuracy while avoiding the need for any manual feature selection. Availability and implementation Code is available at https://github.com/jind11/Deep-PICO-Detection.

scholarly journals Chinese Named Entity Recognition Method in History and Culture Field Based on BERT

2021 ◽  
Vol 14 (1) ◽  
Author(s):  
Shuang Liu ◽  
Hui Yang ◽  
Jiayi Li ◽  
Simon Kolmanič
Keyword(s):  
Short Term Memory ◽  
Conditional Random Field ◽  
Language Model ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Knowledge Graph ◽  
Recognition Method ◽  
Short Term ◽  
Named Entity ◽  
Long Short Term Memory

AbstractWith rapid development of the Internet, people have undergone tremendous changes in the way they obtain information. In recent years, knowledge graph is becoming a popular tool for the public to acquire knowledge. For knowledge graph of Chinese history and culture, most researchers adopted traditional named entity recognition methods to extract entity information from unstructured historical text data. However, the traditional named entity recognition method has certain defects, and it is easy to ignore the association between entities. To extract entities from a large amount of historical and cultural information more accurately and efficiently, this paper proposes one named entity recognition model combining Bidirectional Encoder Representations from Transformers and Bidirectional Long Short-Term Memory-Conditional Random Field (BERT-BiLSTM-CRF). First, a BERT pre-trained language model is used to encode a single character to obtain a vector representation corresponding to each character. Then one Bidirectional Long Short-Term Memory (BiLSTM) layer is applied to semantically encode the input text. Finally, the label with the highest probability is output through the Conditional Random Field (CRF) layer to obtain each character’s category. This model uses the Bidirectional Encoder Representations from Transformers (BERT) pre-trained language model to replace the static word vectors trained in the traditional way. In comparison, the BERT pre-trained language model can dynamically generate semantic vectors according to the context of words, which improves the representation ability of word vectors. The experimental results prove that the model proposed in this paper has achieved excellent results in the task of named entity recognition in the field of historical culture. Compared with the existing named entity identification methods, the precision rate, recall rate, and $$F_1$$ F 1 value have been significantly improved.

scholarly journals Extracting entities with attributes in clinical text via joint deep learning

2019 ◽  
Vol 26 (12) ◽  
pp. 1584-1591 ◽  
Author(s):  
Xue Shi ◽  
Yingping Yi ◽  
Ying Xiong ◽  
Buzhou Tang ◽  
Qingcai Chen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Relation Extraction ◽  
Entity Recognition ◽  
Short Term ◽  
Term Memory ◽  
Clinical Text ◽  
Long Short Term Memory

Abstract Objective Extracting clinical entities and their attributes is a fundamental task of natural language processing (NLP) in the medical domain. This task is typically recognized as 2 sequential subtasks in a pipeline, clinical entity or attribute recognition followed by entity-attribute relation extraction. One problem of pipeline methods is that errors from entity recognition are unavoidably passed to relation extraction. We propose a novel joint deep learning method to recognize clinical entities or attributes and extract entity-attribute relations simultaneously. Materials and Methods The proposed method integrates 2 state-of-the-art methods for named entity recognition and relation extraction, namely bidirectional long short-term memory with conditional random field and bidirectional long short-term memory, into a unified framework. In this method, relation constraints between clinical entities and attributes and weights of the 2 subtasks are also considered simultaneously. We compare the method with other related methods (ie, pipeline methods and other joint deep learning methods) on an existing English corpus from SemEval-2015 and a newly developed Chinese corpus. Results Our proposed method achieves the best F1 of 74.46% on entity recognition and the best F1 of 50.21% on relation extraction on the English corpus, and 89.32% and 88.13% on the Chinese corpora, respectively, which outperform the other methods on both tasks. Conclusions The joint deep learning–based method could improve both entity recognition and relation extraction from clinical text in both English and Chinese, indicating that the approach is promising.

scholarly journals A Semi-supervised Approach for Sentiment Analysis of Arab(ic+izi) Messages: Application to the Algerian Dialect

2021 ◽  
Vol 2 (2) ◽  
Author(s):  
Imane Guellil ◽  
Ahsan Adeel ◽  
Faical Azouaou ◽  
Fodil Benali ◽  
Ala-Eddine Hachani ◽  
...  
Keyword(s):  
Deep Learning ◽  
Sentiment Analysis ◽  
Short Term Memory ◽  
Native Speakers ◽  
State Of The Art ◽  
Research Literature ◽  
Short Term ◽  
Long Short Term Memory ◽  
The One ◽  
Modern Standard

AbstractIn this paper, we propose a semi-supervised approach for sentiment analysis of Arabic and its dialects. This approach is based on a sentiment corpus, constructed automatically and reviewed manually by Algerian dialect native speakers. This approach consists of constructing and applying a set of deep learning algorithms to classify the sentiment of Arabic messages as positive or negative. It was applied on Facebook messages written in Modern Standard Arabic (MSA) as well as in Algerian dialect (DALG, which is a low resourced-dialect, spoken by more than 40 million people) with both scripts Arabic and Arabizi. To handle Arabizi, we consider both options: transliteration (largely used in the research literature for handling Arabizi) and translation (never used in the research literature for handling Arabizi). For highlighting the effectiveness of a semi-supervised approach, we carried out different experiments using both corpora for the training (i.e. the corpus constructed automatically and the one that was reviewed manually). The experiments were done on many test corpora dedicated to MSA/DALG, which were proposed and evaluated in the research literature. Both classifiers are used, shallow and deep learning classifiers such as Random Forest (RF), Logistic Regression(LR) Convolutional Neural Network (CNN) and Long short-term memory (LSTM). These classifiers are combined with word embedding models such as Word2vec and fastText that were used for sentiment classification. Experimental results (F1 score up to 95% for intrinsic experiments and up to 89% for extrinsic experiments) showed that the proposed system outperforms the existing state-of-the-art methodologies (the best improvement is up to 25%).

scholarly journals A Study on Standardization of Security Evaluation Information for Chemical Processes Based on Deep Learning

Processes ◽  
2021 ◽  
Vol 9 (5) ◽  
pp. 832
Author(s):  
Lanfei Peng ◽  
Dong Gao ◽  
Yujie Bai
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Hazard Analysis ◽  
Named Entity Recognition ◽  
Recall Rate ◽  
Entity Recognition ◽  
Language Models ◽  
Security Evaluation ◽  
Long Distance

Hazard and operability analysis (HAZOP) is one of the most commonly used hazard analysis methods in the petrochemical industry. The large amount of unstructured data in HAZOP reports has generated an information explosion which has led to a pressing need for technologies that can simplify the use of this information. In order to solve the problem that massive data are difficult to reuse and share, in this study, we propose a new deep learning framework for Chinese HAZOP documents to perform a named entity recognition (NER) task, aiming at the characteristics of HAZOP documents, such as polysemy, multi-entity nesting, and long-distance text. Specifically, the preprocessed data are input into an embeddings from language models (ELMo) and a double convolutional neural network (DCNN) model to extract rich character features. Meanwhile, a bidirectional long short-term memory (BiLSTM) network is used to extract long-distance semantic information. Finally, the results are decoded by a conditional random field (CRF), and then output. Experiments were carried out using the HAZOP report of a coal seam indirect liquefaction project. The experimental results for the proposed model showed that the accuracy rate of the optimal results reached 90.83, the recall rate reached 92.46, and the F-value reached the highest 91.76%, which was significantly improved as compared with other models.

scholarly journals Towards reliable named entity recognition in the biomedical domain

2019 ◽  
Author(s):  
John Giorgi ◽  
Gary Bader
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
High Performance ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Maximal Increase ◽  
Named Entity ◽  
Task Learning

Motivation: Automatic biomedical named entity recognition (BioNER) is a key task in biomedical information extraction (IE). For some time, state-of-the-art BioNER has been dominated by machine learning methods, particularly conditional random fields (CRFs), with a recent focus on deep learning. However, recent work has suggested that the high performance of CRFs for BioNER may not generalize to corpora other than the one it was trained on. In our analysis, we find that a popular deep learning-based approach to BioNER, known as bidirectional long short-term memory network-conditional random field (BiLSTM-CRF), is correspondingly poor at generalizing - often dramatically overfitting the corpus it was trained on. To address this, we evaluate three modifications of BiLSTM-CRF for BioNER to alleviate overfitting and improve generalization: improved regularization via variational dropout, transfer learning, and multi-task learning. Results: We measure the effect that each strategy has when training/testing on the same corpus ("in-corpus" performance) and when training on one corpus and evaluating on another ("out-of-corpus" performance), our measure of the models ability to generalize. We found that variational dropout improves out-of-corpus performance by an average of 4.62%, transfer learning by 6.48% and multi-task learning by 8.42%. The maximal increase we identified combines multi-task learning and variational dropout, which boosts out-of-corpus performance by 10.75%. Furthermore, we make available a new open-source tool, called Saber, that implements our best BioNER models. Availability: Source code for our biomedical IE tool is available at https://github.com/BaderLab/saber. Corpora and other resources used in this study are available at https://github.com/BaderLab/Towards- reliable-BioNER.

scholarly journals Codon optimization with deep learning to enhance protein expression

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Hongguang Fu ◽  
Yanbing Liang ◽  
Xiuqin Zhong ◽  
ZhiLing Pan ◽  
Lei Huang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Protein Expression ◽  
Dna Sequences ◽  
Short Term Memory ◽  
Codon Optimization ◽  
Conditional Random Field ◽  
Good Method ◽  
Optimization Methods ◽  
Optimization Method ◽  
Adaptation Index

Abstract Heterologous expression is the main approach for recombinant protein production ingenetic synthesis, for which codon optimization is necessary. The existing optimization methods are based on biological indexes. In this paper, we propose a novel codon optimization method based on deep learning. First, we introduce the concept of codon boxes, via which DNA sequences can be recoded into codon box sequences while ignoring the order of bases. Then, the problem of codon optimization can be converted to sequence annotation of corresponding amino acids with codon boxes. The codon optimization models for Escherichia Coli were trained by the Bidirectional Long-Short-Term Memory Conditional Random Field. Theoretically, deep learning is a good method to obtain the distribution characteristics of DNA. In addition to the comparison of the codon adaptation index, protein expression experiments for plasmodium falciparum candidate vaccine and polymerase acidic protein were implemented for comparison with the original sequences and the optimized sequences from Genewiz and ThermoFisher. The results show that our method for enhancing protein expression is efficient and competitive.

A named entity recognition model based on ensemble learning

2020 ◽  
pp. 1-12
Author(s):  
Xinghui Zhu ◽  
Zhuoyang Zou ◽  
Bo Qiao ◽  
Kui Fang ◽  
Yiming Chen
Keyword(s):  
Ensemble Learning ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Knowledge Graph ◽  
Data Set ◽  
Recognition Model ◽  
Named Entity ◽  
Knowledge Graphs

Knowledge Graph has gradually become one of core drivers advancing the Internet and AI in recent years, while there is currently no normal knowledge graph in the field of agriculture. Named Entity Recognition (NER), one important step in constructing knowledge graphs, has become a hot topic in both academia and industry. With the help of the Bidirectional Long Short-Term Memory Network (Bi-LSTM) and Conditional Random Field (CRF) model, we introduce a method of ensemble learning, and implement a named entity recognition model ELER. Our model achieves good results for the CoNLL2003 data set, the accuracy and F1 value in the best experimental results are respectively improved by 1.37% and 0.7% when compared with the BiLSTM-CRF model. In addition, our model achieves an F1 score of 91% for the agricultural data set AgriNER2018, which proves the validity of ELER model for small agriculture sample data sets and lays a foundation for the construction of agricultural knowledge graphs.

Sign in / Sign up

Export Citation Format

Share Document