scholarly journals Applying Citizen Science to Gene, Drug, Disease Relationship Extraction from Biomedical Abstracts

2019 ◽  
Author(s):  
Ginger Tsueng ◽  
Max Nanis ◽  
Jennifer T. Fouquier ◽  
Michael Mayers ◽  
Benjamin M. Good ◽  
...  
Keyword(s):  
Information Extraction ◽  
Citizen Science ◽  
Computational Methods ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Named Entity ◽  
Relationship Extraction ◽  
Biomedical Texts

AbstractBiomedical literature is growing at a rate that outpaces our ability to harness the knowledge contained therein. In order to mine valuable inferences from the large volume of literature, many researchers have turned to information extraction algorithms to harvest information in biomedical texts. Information extraction is usually accomplished via a combination of manual expert curation and computational methods. Advances in computational methods usually depends on the generation of gold standards by a limited number of expert curators. This process can be time consuming and represents an area of biomedical research that is ripe for exploration with citizen science. Citizen scientists have been previously found to be willing and capable of performing named entity recognition of disease mentions in biomedical abstracts, but it was uncertain whether or not the same could be said of relationship extraction. Relationship extraction requires training on identifying named entities as well as a deeper understanding of how different entity types can relate to one another. Here, we used the web-based application Mark2Cure (https://mark2cure.org) to demonstrate that citizen scientists can perform relationship extraction and confirm the importance of accurate named entity recognition on this task. We also discuss opportunities for future improvement of this system, as well as the potential synergies between citizen science, manual biocuration, and natural language processing.

scholarly journals Applying Citizen Science to Gene, Drug, and Disease Relationship Extraction from Biomedical Abstracts

2019 ◽  
Author(s):  
Ginger Tsueng ◽  
Max Nanis ◽  
Jennifer T Fouquier ◽  
Michael Mayers ◽  
Benjamin M Good ◽  
...  
Keyword(s):  
Information Extraction ◽  
Citizen Science ◽  
Computational Methods ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Supplementary Information ◽  
Named Entity ◽  
Relationship Extraction

Abstract Motivation Biomedical literature is growing at a rate that outpaces our ability to harness the knowledge contained therein. To mine valuable inferences from the large volume of literature, many researchers use information extraction algorithms to harvest information in biomedical texts. Information extraction is usually accomplished via a combination of manual expert curation and computational methods. Advances in computational methods usually depends on the time-consuming generation of gold standards by a limited number of expert curators. Citizen science is public participation in scientific research. We previously found that citizen scientists are willing and capable of performing named entity recognition of disease mentions in biomedical abstracts, but did not know if this was true with relationship extraction. Results In this paper, we introduce the Relationship Extraction Module of the web-based application Mark2Cure and demonstrate that citizen scientists can perform relationship extraction. We confirm the importance of accurate named entity recognition on user performance of relationship extraction and identify design issues that impacted data quality. We find that the data generated by citizen scientists can be used to identify relationship types not currently available in the Mark2Cure Relationship Extraction Module. We compare the citizen science-generated data with algorithm-mined data and identify ways in which the two approaches may complement one another. We also discuss opportunities for future improvement of this system, as well as the potential synergies between citizen science, manual biocuration, and natural language processing. Availability Mark2Cure platform: https://mark2cure.org. Mark2Cure source code: https://github.com/sulab/mark2cure Data and analysis code for this paper: https://github.com/gtsueng/M2C_rel_nb Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Citizen Science for Mining the Biomedical Literature

2016 ◽  
Author(s):  
Ginger Tsueng ◽  
Steven M. Nanis ◽  
Jennifer Fouquier ◽  
Benjamin M Good ◽  
Andrew I Su
Keyword(s):  
Information Extraction ◽  
Citizen Science ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Amazon Mechanical Turk ◽  
Biomedical Knowledge ◽  
Named Entity ◽  
Relationship Extraction

I.AbstractBiomedical literature represents one of the largest and fastest growing collections of unstructured biomedical knowledge. Finding critical information buried in the literature can be challenging. In order to extract information from freeflowing text, researchers need to: 1. identify the entities in the text (named entity recognition), 2. apply a standardized vocabulary to these entities (normalization), and 3. identify how entities in the text are related to one another (relationship extraction). Researchers have primarily approached these information extraction tasks through manual expert curation, and computational methods. We have previously demonstrated that named entity recognition (NER) tasks can be crowdsourced to a group of nonexperts via the paid microtask platform, Amazon Mechanical Turk (AMT); and can dramatically reduce the cost and increase the throughput of biocuration efforts. However, given the size of the biomedical literature even information extraction via paid microtask platforms is not scalable. With our web-based application Mark2Cure (http://mark2cure.org), we demonstrate that NER tasks can also be performed by volunteer citizen scientists with high accuracy. We apply metrics from the Zooniverse Matrices of Citizen Science Success and provide the results here to serve as a basis of comparison for other citizen science projects. Further, we discuss design considerations, issues, and the application of analytics for successfully moving a crowdsourcing workflow from a paid microtask platform to a citizen science platform. To our knowledge, this study is the first application of citizen science to a natural language processing task.

scholarly journals Analyzing transfer learning impact in biomedical cross-lingual named entity recognition and normalization

2021 ◽  
Vol 22 (S1) ◽  
Author(s):  
Renzo M. Rivera-Zavala ◽  
Paloma Martínez
Keyword(s):  
Deep Learning ◽  
Recognition Performance ◽  
Named Entity Recognition ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Pharmaceutical Chemical ◽  
Learning Models ◽  
Named Entity ◽  
Entity Identification ◽  
Biomedical Texts

Abstract Background The volume of biomedical literature and clinical data is growing at an exponential rate. Therefore, efficient access to data described in unstructured biomedical texts is a crucial task for the biomedical industry and research. Named Entity Recognition (NER) is the first step for information and knowledge acquisition when we deal with unstructured texts. Recent NER approaches use contextualized word representations as input for a downstream classification task. However, distributed word vectors (embeddings) are very limited in Spanish and even more for the biomedical domain. Methods In this work, we develop several biomedical Spanish word representations, and we introduce two Deep Learning approaches for pharmaceutical, chemical, and other biomedical entities recognition in Spanish clinical case texts and biomedical texts, one based on a Bi-STM-CRF model and the other on a BERT-based architecture. Results Several Spanish biomedical embeddigns together with the two deep learning models were evaluated on the PharmaCoNER and CORD-19 datasets. The PharmaCoNER dataset is composed of a set of Spanish clinical cases annotated with drugs, chemical compounds and pharmacological substances; our extended Bi-LSTM-CRF model obtains an F-score of 85.24% on entity identification and classification and the BERT model obtains an F-score of 88.80% . For the entity normalization task, the extended Bi-LSTM-CRF model achieves an F-score of 72.85% and the BERT model achieves 79.97%. The CORD-19 dataset consists of scholarly articles written in English annotated with biomedical concepts such as disorder, species, chemical or drugs, gene and protein, enzyme and anatomy. Bi-LSTM-CRF model and BERT model obtain an F-measure of 78.23% and 78.86% on entity identification and classification, respectively on the CORD-19 dataset. Conclusion These results prove that deep learning models with in-domain knowledge learned from large-scale datasets highly improve named entity recognition performance. Moreover, contextualized representations help to understand complexities and ambiguity inherent to biomedical texts. Embeddings based on word, concepts, senses, etc. other than those for English are required to improve NER tasks in other languages.

scholarly journals Deep learning with language models improves named entity recognition for PharmaCoNER

2021 ◽  
Vol 22 (S1) ◽  
Author(s):  
Cong Sun ◽  
Zhihao Yang ◽  
Lei Wang ◽  
Yin Zhang ◽  
Hongfei Lin ◽  
...  
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Domain Knowledge ◽  
Named Entity Recognition ◽  
Model Performance ◽  
Relation Extraction ◽  
Entity Recognition ◽  
Language Models ◽  
Named Entity ◽  
Biomedical Texts

Abstract Background The recognition of pharmacological substances, compounds and proteins is essential for biomedical relation extraction, knowledge graph construction, drug discovery, as well as medical question answering. Although considerable efforts have been made to recognize biomedical entities in English texts, to date, only few limited attempts were made to recognize them from biomedical texts in other languages. PharmaCoNER is a named entity recognition challenge to recognize pharmacological entities from Spanish texts. Because there are currently abundant resources in the field of natural language processing, how to leverage these resources to the PharmaCoNER challenge is a meaningful study. Methods Inspired by the success of deep learning with language models, we compare and explore various representative BERT models to promote the development of the PharmaCoNER task. Results The experimental results show that deep learning with language models can effectively improve model performance on the PharmaCoNER dataset. Our method achieves state-of-the-art performance on the PharmaCoNER dataset, with a max F1-score of 92.01%. Conclusion For the BERT models on the PharmaCoNER dataset, biomedical domain knowledge has a greater impact on model performance than the native language (i.e., Spanish). The BERT models can obtain competitive performance by using WordPiece to alleviate the out of vocabulary limitation. The performance on the BERT model can be further improved by constructing a specific vocabulary based on domain knowledge. Moreover, the character case also has a certain impact on model performance.

scholarly journals Combining word embeddings to extract chemical and drug entities in biomedical literature

2021 ◽  
Vol 22 (S1) ◽  
Author(s):  
Pilar López-Úbeda ◽  
Manuel Carlos Díaz-Galiano ◽  
L. Alfonso Ureña-López ◽  
M. Teresa Martín-Valdivia
Keyword(s):  
Language Processing ◽  
Named Entity Recognition ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Levenshtein Distance ◽  
Word Embeddings ◽  
Snomed Ct ◽  
Named Entity ◽  
Indexing System ◽  
The One

Abstract Background Natural language processing (NLP) and text mining technologies for the extraction and indexing of chemical and drug entities are key to improving the access and integration of information from unstructured data such as biomedical literature. Methods In this paper we evaluate two important tasks in NLP: the named entity recognition (NER) and Entity indexing using the SNOMED-CT terminology. For this purpose, we propose a combination of word embeddings in order to improve the results obtained in the PharmaCoNER challenge. Results For the NER task we present a neural network composed of BiLSTM with a CRF sequential layer where different word embeddings are combined as an input to the architecture. A hybrid method combining supervised and unsupervised models is used for the concept indexing task. In the supervised model, we use the training set to find previously trained concepts, and the unsupervised model is based on a 6-step architecture. This architecture uses a dictionary of synonyms and the Levenshtein distance to assign the correct SNOMED-CT code. Conclusion On the one hand, the combination of word embeddings helps to improve the recognition of chemicals and drugs in the biomedical literature. We achieved results of 91.41% for precision, 90.14% for recall, and 90.77% for F1-score using micro-averaging. On the other hand, our indexing system achieves a 92.67% F1-score, 92.44% for recall, and 92.91% for precision. With these results in a final ranking, we would be in the first position.

scholarly journals A Survey on Recent Named Entity Recognition and Relationship Extraction Techniques on Clinical Texts

2021 ◽  
Vol 11 (18) ◽  
pp. 8319
Author(s):  
Priyankar Bose ◽  
Sriram Srinivasan ◽  
William C. Sleeman ◽  
Jatinder Palta ◽  
Rishabh Kapoor ◽  
...  
Keyword(s):  
Information Extraction ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Future Research ◽  
Clinical Text ◽  
Named Entity ◽  
Relationship Extraction ◽  
New Information ◽  
Comprehensive Survey ◽  
Future Research Directions

Significant growth in Electronic Health Records (EHR) over the last decade has provided an abundance of clinical text that is mostly unstructured and untapped. This huge amount of clinical text data has motivated the development of new information extraction and text mining techniques. Named Entity Recognition (NER) and Relationship Extraction (RE) are key components of information extraction tasks in the clinical domain. In this paper, we highlight the present status of clinical NER and RE techniques in detail by discussing the existing proposed NLP models for the two tasks and their performances and discuss the current challenges. Our comprehensive survey on clinical NER and RE encompass current challenges, state-of-the-art practices, and future directions in information extraction from clinical text. This is the first attempt to discuss both of these interrelated topics together in the clinical context. We identified many research articles published based on different approaches and looked at applications of these tasks. We also discuss the evaluation metrics that are used in the literature to measure the effectiveness of the two these NLP methods and future research directions.

scholarly journals Biomedical named entity recognition and linking datasets: survey and our recent development

2020 ◽  
Vol 21 (6) ◽  
pp. 2219-2238 ◽  
Author(s):  
Ming-Siang Huang ◽  
Po-Ting Lai ◽  
Pei-Yen Lin ◽  
Yu-Ting You ◽  
Richard Tzong-Han Tsai ◽  
...  
Keyword(s):  
Language Processing ◽  
Protein Interaction ◽  
Named Entity Recognition ◽  
Paper Analysis ◽  
Biomedical Literature ◽  
Entity Recognition ◽  
Supplementary Information ◽  
Protein Protein Interaction ◽  
Named Entity ◽  
Biomedical Named Entity Recognition

Abstract Natural language processing (NLP) is widely applied in biological domains to retrieve information from publications. Systems to address numerous applications exist, such as biomedical named entity recognition (BNER), named entity normalization (NEN) and protein–protein interaction extraction (PPIE). High-quality datasets can assist the development of robust and reliable systems; however, due to the endless applications and evolving techniques, the annotations of benchmark datasets may become outdated and inappropriate. In this study, we first review commonlyused BNER datasets and their potential annotation problems such as inconsistency and low portability. Then, we introduce a revised version of the JNLPBA dataset that solves potential problems in the original and use state-of-the-art named entity recognition systems to evaluate its portability to different kinds of biomedical literature, including protein–protein interaction and biology events. Lastly, we introduce an ensembled biomedical entity dataset (EBED) by extending the revised JNLPBA dataset with PubMed Central full-text paragraphs, figure captions and patent abstracts. This EBED is a multi-task dataset that covers annotations including gene, disease and chemical entities. In total, it contains 85000 entity mentions, 25000 entity mentions with database identifiers and 5000 attribute tags. To demonstrate the usage of the EBED, we review the BNER track from the AI CUP Biomedical Paper Analysis challenge. Availability: The revised JNLPBA dataset is available at https://iasl-btm.iis.sinica.edu.tw/BNER/Content/Re vised_JNLPBA.zip. The EBED dataset is available at https://iasl-btm.iis.sinica.edu.tw/BNER/Content/AICUP _EBED_dataset.rar. Contact: Email: [email protected], Tel. 886-3-4227151 ext. 35203, Fax: 886-3-422-2681 Email: [email protected], Tel. 886-2-2788-3799 ext. 2211, Fax: 886-2-2782-4814 Supplementary information: Supplementary data are available at Briefings in Bioinformatics online.

scholarly journals ChEMU 2020: Natural Language Processing Methods Are Effective for Information Extraction From Chemical Patents

2021 ◽  
Vol 6 ◽  
Author(s):  
Jiayuan He ◽  
Dat Quoc Nguyen ◽  
Saber A. Akhondi ◽  
Christian Druckenbrodt ◽  
Camilo Thorne ◽  
...  
Keyword(s):  
Information Extraction ◽  
Chemical Reactions ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Event Extraction ◽  
Entity Recognition ◽  
Named Entity ◽  
Fundamental Information ◽  
Essential Chemical ◽  
Reaction Processes

Chemical patents represent a valuable source of information about new chemical compounds, which is critical to the drug discovery process. Automated information extraction over chemical patents is, however, a challenging task due to the large volume of existing patents and the complex linguistic properties of chemical patents. The Cheminformatics Elsevier Melbourne University (ChEMU) evaluation lab 2020, part of the Conference and Labs of the Evaluation Forum 2020 (CLEF2020), was introduced to support the development of advanced text mining techniques for chemical patents. The ChEMU 2020 lab proposed two fundamental information extraction tasks focusing on chemical reaction processes described in chemical patents: (1) chemical named entity recognition, requiring identification of essential chemical entities and their roles in chemical reactions, as well as reaction conditions; and (2) event extraction, which aims at identification of event steps relating the entities involved in chemical reactions. The ChEMU 2020 lab received 37 team registrations and 46 runs. Overall, the performance of submissions for these tasks exceeded our expectations, with the top systems outperforming strong baselines. We further show the methods to be robust to variations in sampling of the test data. We provide a detailed overview of the ChEMU 2020 corpus and its annotation, showing that inter-annotator agreement is very strong. We also present the methods adopted by participants, provide a detailed analysis of their performance, and carefully consider the potential impact of data leakage on interpretation of the results. The ChEMU 2020 Lab has shown the viability of automated methods to support information extraction of key information in chemical patents.

scholarly journals An Annotated Corpus of Crime-Related Portuguese Documents for NLP and Machine Learning Processing

Data ◽  
2021 ◽  
Vol 6 (7) ◽  
pp. 71
Author(s):  
Gonçalo Carnaz ◽  
Mário Antunes ◽  
Vitor Beires Nogueira
Keyword(s):  
Machine Learning ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Automatic Identification ◽  
Named Entities ◽  
Related Data ◽  
Named Entity ◽  
Chain Of Custody ◽  
Evidence Collection

Criminal investigations collect and analyze the facts related to a crime, from which the investigators can deduce evidence to be used in court. It is a multidisciplinary and applied science, which includes interviews, interrogations, evidence collection, preservation of the chain of custody, and other methods and techniques of investigation. These techniques produce both digital and paper documents that have to be carefully analyzed to identify correlations and interactions among suspects, places, license plates, and other entities that are mentioned in the investigation. The computerized processing of these documents is a helping hand to the criminal investigation, as it allows the automatic identification of entities and their relations, being some of which difficult to identify manually. There exists a wide set of dedicated tools, but they have a major limitation: they are unable to process criminal reports in the Portuguese language, as an annotated corpus for that purpose does not exist. This paper presents an annotated corpus, composed of a collection of anonymized crime-related documents, which were extracted from official and open sources. The dataset was produced as the result of an exploratory initiative to collect crime-related data from websites and conditioned-access police reports. The dataset was evaluated and a mean precision of 0.808, recall of 0.722, and F1-score of 0.733 were obtained with the classification of the annotated named-entities present in the crime-related documents. This corpus can be employed to benchmark Machine Learning (ML) and Natural Language Processing (NLP) methods and tools to detect and correlate entities in the documents. Some examples are sentence detection, named-entity recognition, and identification of terms related to the criminal domain.

Sign in / Sign up

Export Citation Format

Share Document