scholarly journals Enhanced Knowledge Graph Embedding by Jointly Learning Soft Rules and Facts

Algorithms ◽  
2019 ◽  
Vol 12 (12) ◽  
pp. 265 ◽  
Author(s):  
Jindou Zhang ◽  
Jing Li
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Background Knowledge ◽  
Background Information ◽  
Knowledge Graph ◽  
Logical Inference ◽  
Training Algorithm ◽  
First Order ◽  
Forward Chaining ◽  
Logical Rules

Combining first order logic rules with a Knowledge Graph (KG) embedding model has recently gained increasing attention, as rules introduce rich background information. Among such studies, models equipped with soft rules, which are extracted with certain confidences, achieve state-of-the-art performance. However, the existing methods either cannot support the transitivity and composition rules or take soft rules as regularization terms to constrain derived facts, which is incapable of encoding the logical background knowledge about facts contained in soft rules. In addition, previous works performed one time logical inference over rules to generate valid groundings for modeling rules, ignoring forward chaining inference, which can further generate more valid groundings to better model rules. To these ends, this paper proposes Soft Logical rules enhanced Embedding (SoLE), a novel KG embedding model equipped with a joint training algorithm over soft rules and KG facts to inject the logical background knowledge of rules into embeddings, as well as forward chaining inference over rules. Evaluations on Freebase and DBpedia show that SoLE not only achieves improvements of 11.6%/5.9% in Mean Reciprocal Rank (MRR) and 18.4%/15.9% in HITS@1 compared to the model on which SoLE is based, but also significantly and consistently outperforms the state-of-the-art baselines in the link prediction task.

scholarly journals TransET: Knowledge Graph Embedding with Entity Types

Electronics ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 1407
Author(s):  
Peng Wang ◽  
Jing Zhou ◽  
Yuzhang Liu ◽  
Xingchen Zhou
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Score Function ◽  
Graph Embedding ◽  
Vector Spaces ◽  
Knowledge Graph ◽  
Semantic Features ◽  
Knowledge Graphs ◽  
Real World Datasets ◽  
Low Dimensional

Knowledge graph embedding aims to embed entities and relations into low-dimensional vector spaces. Most existing methods only focus on triple facts in knowledge graphs. In addition, models based on translation or distance measurement cannot fully represent complex relations. As well-constructed prior knowledge, entity types can be employed to learn the representations of entities and relations. In this paper, we propose a novel knowledge graph embedding model named TransET, which takes advantage of entity types to learn more semantic features. More specifically, circle convolution based on the embeddings of entity and entity types is utilized to map head entity and tail entity to type-specific representations, then translation-based score function is used to learn the presentation triples. We evaluated our model on real-world datasets with two benchmark tasks of link prediction and triple classification. Experimental results demonstrate that it outperforms state-of-the-art models in most cases.

scholarly journals Neighborhood Intervention Consistency: Measuring Confidence for Knowledge Graph Link Prediction

2021 ◽  
Author(s):  
Kai Wang ◽  
Yu Liu ◽  
Quan Z. Sheng
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Confidence Score ◽  
Measurement Methods ◽  
Knowledge Graph ◽  
Critical Gap ◽  
Optimal Score ◽  
Causal Intervention ◽  
Graph Link ◽  
Confidence Measurement

Link prediction based on knowledge graph embeddings (KGE) has recently drawn a considerable momentum. However, existing KGE models suffer from insufficient accuracy and hardly evaluate the confidence probability of each predicted triple. To fill this critical gap, we propose a novel confidence measurement method based on causal intervention, called Neighborhood Intervention Consistency (NIC). Unlike previous confidence measurement methods that focus on the optimal score in a prediction, NIC actively intervenes in the input entity vector to measure the robustness of the prediction result. The experimental results on ten popular KGE models show that our NIC method can effectively estimate the confidence score of each predicted triple. The top 10% triples with high NIC confidence can achieve 30% higher accuracy in the state-of-the-art KGE models.

scholarly journals Contextual Parameter Generation for Knowledge Graph Link Prediction

2020 ◽  
Vol 34 (03) ◽  
pp. 3000-3008
Author(s):  
George Stoica ◽  
Otilia Stretcu ◽  
Emmanouil Antonios Platanios ◽  
Tom Mitchell ◽  
Barnabás Póczos
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Linear Functions ◽  
Convergence Time ◽  
Knowledge Graph ◽  
Complex Interactions ◽  
Significant Performance ◽  
Representational Power ◽  
Graph Link ◽  
Contextual Parameter

We consider the task of knowledge graph link prediction. Given a question consisting of a source entity and a relation (e.g., Shakespeare and BornIn), the objective is to predict the most likely answer entity (e.g., England). Recent approaches tackle this problem by learning entity and relation embeddings. However, they often constrain the relationship between these embeddings to be additive (i.e., the embeddings are concatenated and then processed by a sequence of linear functions and element-wise non-linearities). We show that this type of interaction significantly limits representational power. For example, such models cannot handle cases where a different projection of the source entity is used for each relation. We propose to use contextual parameter generation to address this limitation. More specifically, we treat relations as the context in which source entities are processed to produce predictions, by using relation embeddings to generate the parameters of a model operating over source entity embeddings. This allows models to represent more complex interactions between entities and relations. We apply our method on two existing link prediction methods, including the current state-of-the-art, resulting in significant performance gains and establishing a new state-of-the-art for this task. These gains are achieved while also reducing convergence time by up to 28 times.

A quaternion-group knowledge graph embedding model

2021 ◽  
pp. 1-10
Author(s):  
Heng Chen ◽  
Guanyu Li ◽  
Yunhao Sun ◽  
Wei Jiang
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Graph Embedding ◽  
Knowledge Graph ◽  
Complex Vector ◽  
Quaternion Group ◽  
Rotation Matrices ◽  
Group Knowledge ◽  
Benchmark Datasets ◽  
Complete Relation

Capturing the composite embedding representation of a multi-hop relation path is an extremely vital task in knowledge graph completion. Recently, rotation-based relation embedding models have been widely studied to embed composite relations into complex vector space. However, these models make some over-simplified assumptions on the composite relations, resulting the relations to be commutative. To tackle this problem, this paper proposes a novel knowledge graph embedding model, named QuatGE, which can provide sufficient modeling capabilities for complex composite relations. In particular, our method models each relation as a rotation operator in quaternion group-based space. The advantages of our model are twofold: (1) Since the quaternion group is a non-commutative group (i.e., non-Abelian group), the corresponding rotation matrices of composite relations can be non-commutative; (2) The model has a more expressive setting with stronger modeling capabilities, which is flexible to model and infer the complete relation patterns, including: symmetry/anti-symmetry, inversion and commutative/non-commutative composition. Experimental results on four benchmark datasets show that the proposed method outperforms the existing state-of-the-art models for link prediction, especially on composite relations.

scholarly journals ParamE: Regarding Neural Network Parameters as Relation Embeddings for Knowledge Graph Completion

2020 ◽  
Vol 34 (03) ◽  
pp. 2774-2781
Author(s):  
Feihu Che ◽  
Dawei Zhang ◽  
Jianhua Tao ◽  
Mingyue Niu ◽  
Bocheng Zhao
Keyword(s):  
Neural Network ◽  
Link Prediction ◽  
State Of The Art ◽  
Graph Embedding ◽  
Feature Space ◽  
Knowledge Graph ◽  
Network Parameters ◽  
New Knowledge ◽  
Fitting Ability ◽  
Translational Models

We study the task of learning entity and relation embeddings in knowledge graphs for predicting missing links. Previous translational models on link prediction make use of translational properties but lack enough expressiveness, while the convolution neural network based model (ConvE) takes advantage of the great nonlinearity fitting ability of neural networks but overlooks translational properties. In this paper, we propose a new knowledge graph embedding model called ParamE which can utilize the two advantages together. In ParamE, head entity embeddings, relation embeddings and tail entity embeddings are regarded as the input, parameters and output of a neural network respectively. Since parameters in networks are effective in converting input to output, taking neural network parameters as relation embeddings makes ParamE much more expressive and translational. In addition, the entity and relation embeddings in ParamE are from feature space and parameter space respectively, which is in line with the essence that entities and relations are supposed to be mapped into two different spaces. We evaluate the performances of ParamE on standard FB15k-237 and WN18RR datasets, and experiments show ParamE can significantly outperform existing state-of-the-art models, such as ConvE, SACN, RotatE and D4-STE/Gumbel.

scholarly journals Scalable Rule Learning via Learning Representation

2018 ◽  
Author(s):  
Pouya Ghiasnezhad Omran ◽  
Kewen Wang ◽  
Zhe Wang
Keyword(s):  
Link Prediction ◽  
Sampling Method ◽  
State Of The Art ◽  
Rule Learning ◽  
Representation Learning ◽  
Data Repositories ◽  
New Approach ◽  
First Order ◽  
Learning Rules ◽  
Recent Advancement

We study the problem of learning first-order rules from large Knowledge Graphs (KGs). With recent advancement in information extraction, vast data repositories in the KG format have been obtained such as Freebase and YAGO. However, traditional techniques for rule learning are not scalable for KGs. This paper presents a new approach RLvLR to learning rules from KGs by using the technique of embedding in representation learning together with a new sampling method. Experimental results show that our system outperforms some state-of-the-art systems. Specifically, for massive KGs with hundreds of predicates and over 10M facts, RLvLR is much faster and can learn much more quality rules than major systems for rule learning in KGs such as AMIE+. We also used the RLvLR-mined rules in an inference module to carry out the link prediction task. In this task, RLvLR outperformed Neural LP, a state-of-the-art link prediction system, in both runtime and accuracy.

scholarly journals A Novel Negative Sampling Based on Frequency of Relational Association Entities for Knowledge Graph Embedding

2021 ◽  
Author(s):  
Wanhua Cao ◽  
Yi Zhang ◽  
Juntao Liu ◽  
Ziyun Rao
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Evaluation Criteria ◽  
Relation Extraction ◽  
Graph Embedding ◽  
Semantic Space ◽  
Knowledge Graph ◽  
Knowledge Reasoning ◽  
Relation Prediction ◽  
Low Dimensional

Knowledge graph embedding improves the performance of relation extraction and knowledge reasoning by encoding entities and relationships in low-dimensional semantic space. During training, negative samples are usually constructed by replacing the head/tail entity. And the different replacing relationships lead to different accuracy of the prediction results. This paper develops a negative triplets construction framework according to the frequency of relational association entities. The proposed construction framework can fully consider the quantitative of relations and entities in the dataset to assign the proportion of relation and entity replacement and the frequency of the entities associated with each relationship to set reasonable proportions for different relations. To verify the validity of the proposed construction framework, it is integrated into the state-of-the-art knowledge graph embedding models, such as TransE, TransH, DistMult, ComplEx, and Analogy. And both the evaluation criteria of relation prediction and entity prediction are used to evaluate the performance of link prediction more comprehensively. The experimental results on two commonly used datasets, WN18 and FB15K, show that the proposed method improves entity link and triplet classification accuracy, especially the accuracy of relational link prediction.

scholarly journals Embedding Taxonomical, Situational or Sequential Knowledge Graph Context for Recommendation Tasks

2021 ◽  
Author(s):  
Simon Werner ◽  
Achim Rettinger ◽  
Lavdim Halilaj ◽  
Jürgen Lüttin
Keyword(s):  
Network Architecture ◽  
Internal State ◽  
Background Knowledge ◽  
Background Information ◽  
Knowledge Graph ◽  
Situational Context ◽  
The Matrix ◽  
History Of ◽  
Sequential Information ◽  
Vector Representations

Learned latent vector representations are key to the success of many recommender systems in recent years. However, traditional approaches like matrix factorization produce vector representations that capture global distributions of a static recommendation scenario only. Such latent user or item representations do not capture background knowledge and are not customized to a concrete situational context and the sequential history of events leading up to it. This is a fundamentally limiting restriction for many tasks and applications, since the latent state can depend on a) abstract background information, b) the current situational context and c) the history of related observations. An illustrating example is a restaurant recommendation scenario, where a user’s assessment of the situation depends a) on taxonomical information regarding the type of cuisine, b) on situational factors like time of day, weather or location and c) on the subjective individual history and experience of this user in preceding situations. This situation-specific internal state of the user is not captured when using a traditional collaborative filtering approach, since background knowledge, the situational context and the sequential nature of an individual’s history cannot easily be represented in the matrix. In this paper, we investigate how well state-of-the-art approaches do exploit those different dimensions relevant to POI recommendation tasks. Naturally, we represent such a scenario as a temporal knowledge graph and compare plain knowledge graph, a taxonomy and a hypergraph embedding approach, as well as a recurrent neural network architecture to exploit the different context-dimensions of such rich information. Our empirical evidence indicates that the situational context is most crucial to the prediction performance, while the taxonomical and sequential information are harder to exploit. However, they still have their specific merits depending on the situation.

scholarly journals Dual Quaternion Embeddings for Link Prediction

2021 ◽  
Vol 11 (12) ◽  
pp. 5572
Author(s):  
Liming Gao ◽  
Huiling Zhu ◽  
Hankz Hankui Zhuo ◽  
Jin Xu 
Keyword(s):  
Artificial Intelligence ◽  
Link Prediction ◽  
State Of The Art ◽  
Knowledge Graph ◽  
Dual Quaternion ◽  
Novel Approach ◽  
Quaternion Space ◽  
Knowledge Graphs ◽  
Low Dimensional

The applications of knowledge graph have received much attention in the field of artificial intelligence. The quality of knowledge graphs is, however, often influenced by missing facts. To predict the missing facts, various solid transformation based models have been proposed by mapping knowledge graphs into low dimensional spaces. However, most of the existing transformation based approaches ignore that there are multiple relations between two entities, which is common in the real world. In order to address this challenge, we propose a novel approach called DualQuatE that maps entities and relations into a dual quaternion space. Specifically, entities are represented by pure quaternions and relations are modeled based on the combination of rotation and translation from head to tail entities. After that we utilize interactions of different translations and rotations to distinguish various relations between head and tail entities. Experimental results exhibit that the performance of DualQuatE is competitive compared to the existing state-of-the-art models.

scholarly journals Anytime Bottom-Up Rule Learning for Knowledge Graph Completion

2019 ◽  
Author(s):  
Christian Meilicke ◽  
Melisachew Wudage Chekol ◽  
Daniel Ruffinelli ◽  
Heiner Stuckenschmidt
Keyword(s):  
State Of The Art ◽  
Rule Learning ◽  
Knowledge Graph ◽  
Bottom Up ◽  
Rule Based ◽  
Current State ◽  
Knowledge Graphs ◽  
Latent Representations ◽  
Computational Resources ◽  
Logical Rules

We propose an anytime bottom-up technique for learning logical rules from large knowledge graphs. We apply the learned rules to predict candidates in the context of knowledge graph completion. Our approach outperforms other rule-based approaches and it is competitive with current state of the art, which is based on latent representations. Besides, our approach is significantly faster, requires less computational resources, and yields an explanation in terms of the rules that propose a candidate.

Sign in / Sign up

Export Citation Format

Share Document