scholarly journals A General Method for Transferring Explicit Knowledge into Language Model Pretraining

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Ruiqing Yan ◽  
Lanchang Sun ◽  
Fang Wang ◽  
Xiaoming Zhang
Keyword(s):  
Domain Knowledge ◽  
Explicit Knowledge ◽  
Language Model ◽  
Background Knowledge ◽  
Knowledge Bases ◽  
Language Models ◽  
Domain Specific ◽  
Text Understanding ◽  
Domain Specific Knowledge ◽  
General Method

Recently, pretrained language models, such as Bert and XLNet, have rapidly advanced the state of the art on many NLP tasks. They can model implicit semantic information between words in the text. However, it is solely at the token level without considering the background knowledge. Intuitively, background knowledge influences the efficacy of text understanding. Inspired by this, we focus on improving model pretraining by leveraging external knowledge. Different from recent research that optimizes pretraining models by knowledge masking strategies, we propose a simple but general method to transfer explicit knowledge with pretraining. To be specific, we first match knowledge facts from a knowledge base (KB) and then add a knowledge injunction layer to a transformer directly without changing its architecture. This study seeks to find the direct impact of explicit knowledge on model pretraining. We conduct experiments on 7 datasets using 5 knowledge bases in different downstream tasks. Our investigation reveals promising results in all the tasks. The experiment also verifies that domain-specific knowledge is superior to open-domain knowledge in domain-specific task, and different knowledge bases have different performances in different tasks.

scholarly journals Cross-Fertilizing Deep Web Analysis and Ontology Enrichment

2017 ◽  
Author(s):  
Marilena Oita ◽  
Antoine Amarilli ◽  
Pierre Senellart
Keyword(s):  
Domain Knowledge ◽  
Deep Web ◽  
Web Pages ◽  
Complete Understanding ◽  
Specific Knowledge ◽  
Domain Specific ◽  
Domain Specific Knowledge ◽  
Web Crawlers ◽  
New Perspective ◽  
The Impact

Deep Web databases, whose content is presented as dynamically-generated Web pages hidden behind forms, have mostly been left unindexed by search engine crawlers. In order to automatically explore this mass of information, many current techniques assume the existence of domain knowledge, which is costly to create and maintain. In this article, we present a new perspective on form understanding and deep Web data acquisition that does not require any domain-specific knowledge. Unlike previous approaches, we do not perform the various steps in the process (e.g., form understanding, record identification, attribute labeling) independently but integrate them to achieve a more complete understanding of deep Web sources. Through information extraction techniques and using the form itself for validation, we reconcile input and output schemas in a labeled graph which is further aligned with a generic ontology. The impact of this alignment is threefold: first, the resulting semantic infrastructure associated with the form can assist Web crawlers when probing the form for content indexing; second, attributes of response pages are labeled by matching known ontology instances, and relations between attributes are uncovered; and third, we enrich the generic ontology with facts from the deep Web.

The REA Pattern, Knowledge Structures, and Conceptual Modeling Performance

2005 ◽  
Vol 19 (2) ◽  
pp. 57-77 ◽  
Author(s):  
Gregory J. Gerard
Keyword(s):  
Domain Knowledge ◽  
Conceptual Models ◽  
Conceptual Modeling ◽  
Knowledge Structure ◽  
Specific Pattern ◽  
Accounting Information Systems ◽  
Domain Specific ◽  
Domain Specific Knowledge ◽  
Business Setting ◽  
Structured Knowledge

Most database textbooks on conceptual modeling do not cover domainspecific patterns. The texts emphasize notation, apparently assuming that notation enables individuals to correctly model domain-specific knowledge acquired from experience. However, the domain knowledge acquired may not aid in the construction of conceptual models if it is not structured to support conceptual modeling. This study uses the Resources Events Agents (REA) pattern as an example of a domain-specific pattern that can be encoded as a knowledge structure for conceptual modeling of accounting information systems (AIS), and tests its effects on the accuracy of conceptual modeling in a familiar business setting. Fifty-three undergraduate and forty-six graduate students completed recall tasks designed to measure REA knowledge structure. The accuracy of participants' conceptual models was positively related to REA knowledge structure. Results suggest it is insufficient to know only conceptual modeling notation because structured knowledge of domain-specific patterns reduces design errors.

Incremental Ontology Population and Enrichment through Semantic-based Text Mining

2015 ◽  
Vol 11 (3) ◽  
pp. 44-66 ◽  
Author(s):  
Saira Gillani ◽  
Andrea Ko
Keyword(s):  
Text Mining ◽  
Domain Knowledge ◽  
Learning System ◽  
Domain Specific ◽  
Automatic Categorization ◽  
New Concepts ◽  
Domain Specific Knowledge ◽  
E Learning ◽  
Ontology Population ◽  
It Audit

Higher education and professional trainings often apply innovative e-learning systems, where ontologies are used for structuring domain knowledge. To provide up-to-date knowledge for the students, ontology has to be maintained regularly. It is especially true for IT audit and security domain, because technology is changing fast. However manual ontology population and enrichment is a complex task that require professional experience involving a lot of efforts. The authors' paper deals with the challenges and possible solutions for semi-automatic ontology enrichment and population. ProMine has two main contributions; one is the semantic-based text mining approach for automatically identifying domain-specific knowledge elements; the other is the automatic categorization of these extracted knowledge elements by using Wiktionary. ProMine ontology enrichment solution was applied in IT audit domain of an e-learning system. After ten cycles of the application ProMine, the number of automatically identified new concepts are tripled and ProMine categorized new concepts with high precision and recall.

scholarly journals Integrating functional synthesis

2005 ◽  
Vol 19 (3) ◽  
pp. 183-200
Author(s):  
WILLIAM H. WOOD ◽  
HUI DONG ◽  
CLIVE L. DYM
Keyword(s):  
Integrated Design ◽  
Knowledge Bases ◽  
Design Environment ◽  
Simulation Tools ◽  
Domain Specific ◽  
Computational Synthesis ◽  
Domain Specific Knowledge ◽  
Design Function ◽  
Comprehensive Search ◽  
Abstraction Levels

Design couples synthesis and analysis in iterative cycles, alternatively generating solutions, and evaluating their validity. The accuracy and depth of evaluation has increased markedly because of the availability of powerful simulation tools and the development of domain-specific knowledge bases. Efforts to extend the state of the art in evaluation have unfortunately been carried out in stovepipe fashion, depending on domain-specific views both of function and of what constitutes “good” design. Although synthesis as practiced by humans is an intentional process that centers on the notion of function, computational synthesis often eschews such intention for sheer permutation. Rather than combining synthesis and analysis to form an integrated design environment, current methods focus on comprehensive search for solutions within highly circumscribed subdomains of design. This paper presents an overview of the progress made in representing design function across abstraction levels proven useful to human designers. Through an example application in the domain of mechatronics, these representations are integrated across domains and throughout the design process.

Research on feature-based opinion mining using topic maps

2016 ◽  
Vol 34 (3) ◽  
pp. 435-456 ◽  
Author(s):  
Lixin Xia ◽  
Zhongyi Wang ◽  
Chen Chen ◽  
Shanshan Zhai
Keyword(s):  
Domain Knowledge ◽  
Opinion Mining ◽  
Specific Knowledge ◽  
Content Type ◽  
Topic Maps ◽  
Domain Specific ◽  
Product Features ◽  
Domain Specific Knowledge ◽  
Feature Based ◽  
Integrate Domain

Purpose Opinion mining (OM), also known as “sentiment classification”, which aims to discover common patterns of user opinions from their textual statements automatically or semi-automatically, is not only useful for customers, but also for manufacturers. However, because of the complexity of natural language, there are still some problems, such as domain dependence of sentiment words, extraction of implicit features and others. The purpose of this paper is to propose an OM method based on topic maps to solve these problems. Design/methodology/approach Domain-specific knowledge is key to solve problems in feature-based OM. On the one hand, topic maps, as an ontology framework, are composed of topics, associations, occurrences and scopes, and can represent a class of knowledge representation schemes. On the other hand, compared with ontology, topic maps have many advantages. Thus, it is better to integrate domain-specific knowledge into OM based on topic maps. This method can make full use of the semantic relationships among feature words and sentiment words. Findings In feature-level OM, most of the existing research associate product features and opinions by their explicit co-occurrence, or use syntax parsing to judge the modification relationship between opinion words and product features within a review unit. They are mostly based on the structure of language units without considering domain knowledge. Only few methods based on ontology incorporate domain knowledge into feature-based OM, but they only use the “is-a” relation between concepts. Therefore, this paper proposes feature-based OM using topic maps. The experimental results revealed that this method can improve the accuracy of the OM. The findings of this study not only advance the state of OM research but also shed light on future research directions. Research limitations/implications To demonstrate the “feature-based OM using topic maps” applications, this work implements a prototype that helps users to find their new washing machines. Originality/value This paper presents a new method of feature-based OM using topic maps, which can integrate domain-specific knowledge into feature-based OM effectively. This method can improve the accuracy of the OM greatly. The proposed method can be applied across various application domains, such as e-commerce and e-government.

A KNOWLEDGE LEARNING APPROACH TO INFORMATION EXTRACTION FROM MULTIPLE TEXT BASED WEB SITES

2004 ◽  
Vol 13 (03) ◽  
pp. 721-738 ◽  
Author(s):  
XIAOYING GAO ◽  
MENGJIE ZHANG
Keyword(s):  
Information Extraction ◽  
Web Sites ◽  
Learning Algorithm ◽  
Knowledge Bases ◽  
Web Pages ◽  
Specific Knowledge ◽  
Domain Specific ◽  
Domain Specific Knowledge ◽  
Knowledge Unit ◽  
Training Examples

This paper describes a learning/adaptive approach to automatically building knowledge bases for information extraction from text based web pages. A frame based representation is introduced to represent domain knowledge as knowledge unit frames. A frame learning algorithm is developed to automatically learn knowledge unit frames from training examples. Some training examples can be obtained by automatically parsing a number of tabular web pages in the same domain, which greatly reduced the amount of time consuming manual work. This approach was investigated on ten web sites of real estate advertisements and car advertisements and nearly all the information was successfully extracted with very few false alarms. These results suggest that both the knowledge unit frame representation and the frame learning algorithm work well, domain specific knowledge bases can be learned from training examples, and the domain specific knowledge base can be used for information extraction from flexible text-based semi-structured Web pages on multiple Web sites. The investigation of the knowledge representation on five other domains suggests that this approach can be easily applied to other domains by simply changing the training examples.

Workflows Científicos com Apoio de Bases de Conhecimento em Tempo Real

2020 ◽  
Author(s):  
Victor S. Bursztyn ◽  
Jonas Dias ◽  
Marta Mattoso
Keyword(s):  
Knowledge Base ◽  
Domain Knowledge ◽  
Large Scale ◽  
Workflow Engine ◽  
Workflow Execution ◽  
Human In The Loop ◽  
Domain Specific ◽  
Political Sciences ◽  
Provenance Data ◽  
Domain Specific Knowledge

One major challenge in large-scale experiments is the analytical capacity to contrast ongoing results with domain knowledge. We approach this challenge by constructing a domain-specific knowledge base, which is queried during workflow execution. We introduce K-Chiron, an integrated solution that combines a state-of-the-art automatic knowledge base construction (KBC) system to Chiron, a well-established workflow engine. In this work we experiment in the context of Political Sciences to show how KBC may be used to improve human-in-the-loop (HIL) support in scientific experiments. While HIL in traditional domain expert supervision is done offline, in K-Chiron it is done online, i.e. at runtime. We achieve results in less laborious ways, to the point of enabling a breed of experiments that could be unfeasible with traditional HIL. Finally, we show how provenance data could be leveraged with KBC to enable further experimentation in more dynamic settings.

Sign in / Sign up

Export Citation Format

Share Document