scholarly journals Annotation Guideline No. 4: Annotating Narrative Levels in Literature

2019 ◽  
Author(s):  
Nora Ketschik ◽  
Sandra Murr ◽  
Yvonne Zimmermann
Keyword(s):  
Annotation Guideline

Deep Level Analysis of Legitimacy in Bengali News Sentences

2022 ◽  
Vol 21 (1) ◽  
pp. 1-18
Author(s):  
Soma Das ◽  
Pooja Rai ◽  
Sanjay Chatterji
Keyword(s):  
Deep Level ◽  
Research Direction ◽  
Machine Learning Algorithms ◽  
Supervised Machine Learning ◽  
News Article ◽  
Support Vector ◽  
Potential Threat ◽  
Annotation Guideline ◽  
Level 1 ◽  
Level 2

The tremendous increase in the growth of misinformation in news articles has the potential threat for the adverse effects on society. Hence, the detection of misinformation in news data has become an appealing research area. The task of annotating and detecting distorted news article sentences is the immediate need in this research direction. Therefore, an attempt has been made to formulate the legitimacy annotation guideline followed by annotation and detection of the legitimacy in Bengali e-papers. The sentence-level manual annotation of Bengali news has been carried out in two levels, namely “Level-1 Shallow Level Classification” and “Level-2 Deep Level Classification” based on semantic properties of Bengali sentences. The tagging of 1,300 anonymous Bengali e-paper sentences has been done using the formulated guideline-based tags for both levels. The validation of the annotation guideline has been done by applying benchmark supervised machine learning algorithms using the lexical feature, syntactic feature, domain-specific feature, and Level-2 specific feature in both levels. Performance evaluation of these classifiers is done in terms of Accuracy, Precision, Recall, and F-Measure. In both levels, Support Vector Machine outperforms other benchmark classifiers with an accuracy of 72% and 65% in Level-1 and Level-2, respectively.

scholarly journals Temporal Annotation in the Clinical Domain

2014 ◽  
Vol 2 ◽  
pp. 143-154 ◽  
Author(s):  
William F. Styler ◽  
Steven Bethard ◽  
Sean Finan ◽  
Martha Palmer ◽  
Sameer Pradhan ◽  
...  
Keyword(s):  
Formal Specification ◽  
Temporal Information ◽  
Clinical Notes ◽  
Annotation Guideline ◽  
Temporal Annotation ◽  
Clinical Domain

This article discusses the requirements of a formal specification for the annotation of temporal information in clinical narratives. We discuss the implementation and extension of ISO-TimeML for annotating a corpus of clinical notes, known as the THYME corpus. To reflect the information task and the heavily inference-based reasoning demands in the domain, a new annotation guideline has been developed, “the THYME Guidelines to ISO-TimeML (THYME-TimeML)”. To clarify what relations merit annotation, we distinguish between linguistically-derived and inferentially-derived temporal orderings in the text. We also apply a top performing TempEval 2013 system against this new resource to measure the difficulty of adapting systems to the clinical domain. The corpus is available to the community and has been proposed for use in a SemEval 2015 task.

scholarly journals Deep learning models in detection of dietary supplement adverse event signals from Twitter

JAMIA Open ◽  
2021 ◽  
Vol 4 (4) ◽  
Author(s):  
Yefeng Wang ◽  
Yunpeng Zhao ◽  
Dalton Schutte ◽  
Jiang Bian ◽  
Rui Zhang
Keyword(s):  
Adverse Event ◽  
Deep Learning ◽  
Knowledge Base ◽  
Dietary Supplement ◽  
Relation Extraction ◽  
Learning Models ◽  
Concept Extraction ◽  
Annotation Guideline ◽  
End To End ◽  
Ae Signals

Abstract Objective The objective of this study is to develop a deep learning pipeline to detect signals on dietary supplement-related adverse events (DS AEs) from Twitter. Materials and Methods We obtained 247 807 tweets ranging from 2012 to 2018 that mentioned both DS and AE. We designed a tailor-made annotation guideline for DS AEs and annotated biomedical entities and relations on 2000 tweets. For the concept extraction task, we fine-tuned and compared the performance of BioClinical-BERT, PubMedBERT, ELECTRA, RoBERTa, and DeBERTa models with a CRF classifier. For the relation extraction task, we fine-tuned and compared BERT models to BioClinical-BERT, PubMedBERT, RoBERTa, and DeBERTa models. We chose the best-performing models in each task to assemble an end-to-end deep learning pipeline to detect DS AE signals and compared the results to the known DS AEs from a DS knowledge base (ie, iDISK). Results DeBERTa-CRF model outperformed other models in the concept extraction task, scoring a lenient microaveraged F1 score of 0.866. RoBERTa model outperformed other models in the relation extraction task, scoring a lenient microaveraged F1 score of 0.788. The end-to-end pipeline built on these 2 models was able to extract DS indication and DS AEs with a lenient microaveraged F1 score of 0.666. Conclusion We have developed a deep learning pipeline that can detect DS AE signals from Twitter. We have found DS AEs that were not recorded in an existing knowledge base (iDISK) and our proposed pipeline can as sist DS AE pharmacovigilance.

Sign in / Sign up

Export Citation Format

Share Document