In-the-Moment Visual Information from the Infant's Egocentric View Determines the Success of Infant Word Learning: A Computational Study

Infants learn the meaning of words from accumulated experiences of real-time interactions with their caregivers. To study the effects of visual sensory input on word learning, we recorded infant's view of the world using head-mounted eye trackers during free-flowing play with a caregiver. While playing, infants were exposed to novel label-object mappings and later learning outcomes for these items were tested after the play session. In this study we use a classification based approach to link properties of infants' visual scenes during naturalistic labeling moments to their word learning outcomes. We find that a model which integrates both highly informative and ambiguous sensory evidence is a better fit to infants' individual learning outcomes than models where either type of evidence is taken alone, and that raw labeling frequency is unable to account for the word learning differences we observe. Here we demonstrate how a computational model, using only raw pixels taken from the egocentric scene image, can derive insights on human language learning.

EXPRESS: The unique role of novel linguistic labels on the disengagement of visual attention

Do novel linguistic labels have privileged access to attentional resources compared to non-linguistic labels? This study explores this possibility through two experiments with a training and an attentional overlap task. Experiment 1 investigates how novel label and object-only stimuli influence resource allocation and disengagement of visual attention. Experiment 2 tests the impact of linguistic information on visual attention by comparing novel tones and labels. Because disengagement of attention is affected both by the saliency of the perceptual stimulus and by the degree of familiarity with the stimulus to be disengaged from, we compared pupil size variations and saccade latency under different test conditions: (i) consistent with (i.e., identical to) the training; (ii) inconsistent with the training (i.e., with an altered feature), and (iii) deprived of one feature only in Experiment 1. Experiment 1 indicated a general consistency advantage (and deprived disadvantage) driven by linguistic label-object pairs compared to object-only stimuli. Experiment 2 revealed that tone-object pairs led to higher pupil dilation and longer saccade latency than linguistic label-object pairs. Our results suggest that novel linguistic labels preferentially impact the early orienting of attention.

Hierarchical Multi-Label Object Detection Framework for Remote Sensing Images

Detecting objects such as aircraft and ships is a fundamental research area in remote sensing analytics. Owing to the prosperity and development of CNNs, many previous methodologies have been proposed for object detection within remote sensing images. Despite the advance, using the object detection datasets with a more complex structure, i.e., datasets with hierarchically multi-labeled objects, is limited to the existing detection models. Especially in remote sensing images, since objects are obtained from bird’s-eye view, the objects are captured with restricted visual features and not always guaranteed to be labeled up to fine categories. We propose a hierarchical multi-label object detection framework applicable to hierarchically partial-annotated datasets. In the framework, an object detection pipeline called Decoupled Hierarchical Classification Refinement (DHCR) fuses the results of two networks: (1) an object detection network with multiple classifiers, and (2) a hierarchical sibling classification network for supporting hierarchical multi-label classification. Our framework additionally introduces a region proposal method for efficient detection on vain areas of the remote sensing images, called clustering-guided cropping strategy. Thorough experiments validate the effectiveness of our framework on our own object detection datasets constructed with remote sensing images from WorldView-3 and SkySat satellites. Under our proposed framework, DHCR-based detections significantly improve the performance of respective baseline models and we achieve state-of-the-art results on the datasets.

Do children go for the nice guys? The influence of speaker benevolence and certainty on selective word learning

This study investigated how speaker certainty (a rational cue) and speaker benevolence (an emotional cue) influence children's willingness to learn words in a selective learning paradigm. In two experiments four- to six-year-olds learnt novel labels from two speakers and, after a week, their memory for these labels was reassessed. Results demonstrated that children retained the label–object pairings for at least a week. Furthermore, children preferred to learn from certain over uncertain speakers, but they had no significant preference for nice over nasty speakers. When the cues were combined, children followed certain speakers, even if they were nasty. However, children did prefer to learn from nice and certain speakers over nasty and certain speakers. These results suggest that rational cues regarding a speaker's linguistic competence trump emotional cues regarding a speaker's affective status in word learning. However, emotional cues were found to have a subtle influence on this process.

Children’s referent selection and word learning

It is well-established that toddlers can correctly select a novel referent from an ambiguous array in response to a novel label. There is also a growing consensus that robust word learning requires repeated label-object encounters. However, the effect of the context in which a novel object is encountered is less well-understood. We present two embodied neural network replications of recent empirical tasks, which demonstrated that the context in which a target object is encountered is fundamental to referent selection and word learning. Our model offers an explicit account of the bottom-up associative and embodied mechanisms which could support children’s early word learning and emphasises the importance of viewing behaviour as the interaction of learning at multiple timescales.