Perturb, Predict & Paraphrase: Semi-Supervised Learning using Noisy Student for Image Captioning

Recent semi-supervised learning (SSL) methods are predominantly focused on multi-class classification tasks. Classification tasks allow for easy mixing of class labels during augmentation which does not trivially extend to structured outputs such as word sequences that appear in tasks like image captioning. Noisy Student Training is a recent SSL paradigm proposed for image classification that is an extension of self-training and teacher-student learning. In this work, we provide an in-depth analysis of the noisy student SSL framework for the task of image captioning and derive state-of-the-art results. The original algorithm relies on computationally expensive data augmentation steps that involve perturbing the raw images and computing features for each perturbed image. We show that, even in the absence of raw image augmentation, the use of simple model and feature perturbations to the input images for the student model are beneficial to SSL training. We also show how a paraphrase generator could be effectively used for label augmentation to improve the quality of pseudo labels and significantly improve performance. Our final results in the limited labeled data setting (1% of the MS-COCO labeled data) outperform previous state-of-the-art approaches by 2.5 on BLEU4 and 11.5 on CIDEr scores.

Knowledge translation: a case study in a public health network in Manguinhos, Brazil

Background This paper presents a post hoc analysis of knowledge translation (KT) actions and strategies implemented by three projects in a sociotechnical network, in Rio de Janeiro, Brazil. In order to assess the actions and practices of knowledge producers (mostly researchers) and knowledge users (residents of Manguinhos) we applied the KT model developed by the Québec Public Health Institute. Methods This case study relied mainly on document analysis (texts produced by the network coordination, meeting minutes and reports, management reports and promotional material), interviews with knowledge producers (N = 10), and focus group with knowledge users (4 participants). Framework analysis was applied to provide clear steps to follow and structured outputs of summarized data. A content analysis of this material used categories such as: project development; KT products elaboration; and interaction between knowledge producers and users. Data were coded based on the KT model to understand whether and how the eight dimensions were implemented. Results The findings reveal that, albeit there were differences among the three cases, the KT dimensions related to the co-construction of knowledge, what to be translated, and how to translate were more extensively implemented. Even though KT was a new concept for most knowledge producers, all three cases had previous practical experience on how to disseminate knowledge in the Territory of Manguinhos. However, dimensions related to KT evaluation and resources were less frequently implemented. Conclusions More attention must be paid to the dimensions involving the feasibility, resources and evaluation of projects. Creating research organizations working together in the KT process with support, infrastructure, theoretical and methodological competences about KT may facilitate the integration of these dimensions. Key messages Through our study, we provide more evidence and progress about how the KT process can be improved in low- and middle-income countries, such as Brazil. We display an overview of the challenges that public health researchers in Brazil have in applying KT strategies to improve the public health care.

Adversarial Constraint Learning for Structured Prediction

Constraint-based learning reduces the burden of collecting labels by having users specify general properties of structured outputs, such as constraints imposed by physical laws. We propose a novel framework for simultaneously learning these constraints and using them for supervision, bypassing the difficulty of using domain expertise to manually specify constraints. Learning requires a black-box simulator of structured outputs, which generates valid labels, but need not model their corresponding inputs or the input-label relationship. At training time, we constrain the model to produce outputs that cannot be distinguished from simulated labels by adversarial training. Providing our framework with a small number of labeled inputs gives rise to a new semi-supervised structured prediction model; we evaluate this model on multiple tasks --- tracking, pose estimation and time series prediction --- and find that it achieves high accuracy with only a small number of labeled inputs. In some cases, no labels are required at all.

HC-Search: A Learning Framework for Search-based Structured Prediction

Structured prediction is the problem of learning a function that maps structured inputs to structured outputs. Prototypical examples of structured prediction include part-of-speech tagging and semantic segmentation of images. Inspired by the recent successes of search-based structured prediction, we introduce a new framework for structured prediction called HC-Search. Given a structured input, the framework uses a search procedure guided by a learned heuristic H to uncover high quality candidate outputs and then employs a separate learned cost function C to select a final prediction among those outputs. The overall loss of this prediction architecture decomposes into the loss due to H not leading to high quality outputs, and the loss due to C not selecting the best among the generated outputs. Guided by this decomposition, we minimize the overall loss in a greedy stage-wise manner by first training H to quickly uncover high quality outputs via imitation learning, and then training C to correctly rank the outputs generated via H according to their true losses. Importantly, this training procedure is sensitive to the particular loss function of interest and the time-bound allowed for predictions. Experiments on several benchmark domains show that our approach significantly outperforms several state-of-the-art methods.