Weakly-Supervised Structured Output Learning with Flexible and Latent Graphs Using High-Order Loss Functions

2015 ◽  
Author(s):  
Gustavo Carneiro ◽  
Tingying Peng ◽  
Christine Bayer ◽  
Nassir Navab
Keyword(s):  
High Order ◽  
Loss Functions ◽  
Structured Output Learning ◽  
Structured Output ◽  
Weakly Supervised

scholarly journals Inference-Masked Loss for Deep Structured Output Learning

2020 ◽  
Author(s):  
Quan Guo ◽  
Hossein Rajaby Faghihi ◽  
Yue Zhang ◽  
Andrzej Uszok ◽  
Parisa Kordjamshidi
Keyword(s):  
Domain Knowledge ◽  
Deep Neural Networks ◽  
Loss Functions ◽  
Structured Learning ◽  
Local Error ◽  
Structured Output Learning ◽  
Output Constraints ◽  
Log Likelihood ◽  
Gradient Based ◽  
Structured Output

Structured learning algorithms usually involve an inference phase that selects the best global output variables assignments based on the local scores of all possible assignments. We extend deep neural networks with structured learning to combine the power of learning representations and leveraging the use of domain knowledge in the form of output constraints during training. Introducing a non-differentiable inference module to gradient-based training is a critical challenge. Compared to using conventional loss functions that penalize every local error independently, we propose an inference-masked loss that takes into account the effect of inference and does not penalize the local errors that can be corrected by the inference. We empirically show the inference-masked loss combined with the negative log-likelihood loss improves the performance on different tasks, namely entity relation recognition on CoNLL04 and ACE2005 corpora, and spatial role labeling on CLEF 2017 mSpRL dataset. We show the proposed approach helps to achieve better generalizability, particularly in the low-data regime.

scholarly journals Flexible and Latent Structured Output Learning

2015 ◽  
pp. 220-228 ◽  
Author(s):  
Gustavo Carneiro ◽  
Tingying Peng ◽  
Christine Bayer ◽  
Nassir Navab
Keyword(s):  
Structured Output Learning ◽  
Structured Output

scholarly journals Structured Output Learning with Conditional Generative Flows

2020 ◽  
Vol 34 (04) ◽  
pp. 5005-5012 ◽  
Author(s):  
You Lu ◽  
Bert Huang
Keyword(s):  
Prediction Models ◽  
State Of The Art ◽  
Prediction Method ◽  
Structured Prediction ◽  
Conditional Likelihood ◽  
Structured Output Learning ◽  
Approximate Likelihood ◽  
Structured Output ◽  
Prediction Problems ◽  
The Relationship

Traditional structured prediction models try to learn the conditional likelihood, i.e., p(y|x), to capture the relationship between the structured output y and the input features x. For many models, computing the likelihood is intractable. These models are therefore hard to train, requiring the use of surrogate objectives or variational inference to approximate likelihood. In this paper, we propose conditional Glow (c-Glow), a conditional generative flow for structured output learning. C-Glow benefits from the ability of flow-based models to compute p(y|x exactly and efficiently. Learning with c-Glow does not require a surrogate objective or performing inference during training. Once trained, we can directly and efficiently generate conditional samples. We develop a sample-based prediction method, which can use this advantage to do efficient and effective inference. In our experiments, we test c-Glow on five different tasks. C-Glow outperforms the state-of-the-art baselines in some tasks and predicts comparable outputs in the other tasks. The results show that c-Glow is versatile and is applicable to many different structured prediction problems.

Sign in / Sign up

Export Citation Format

Share Document