scholarly journals Jointly Learning to Parse and Perceive: Connecting Natural Language to the Physical World

2013 ◽  
Vol 1 ◽  
pp. 193-206 ◽  
Author(s):  
Jayant Krishnamurthy ◽  
Thomas Kollar
Keyword(s):  
Language Acquisition ◽  
Natural Language ◽  
Physical Environment ◽  
Question Answering ◽  
Scene Understanding ◽  
Physical World ◽  
Training Procedure ◽  
Supervised Training ◽  
Weakly Supervised ◽  
Logical Semantics

This paper introduces Logical Semantics with Perception (LSP), a model for grounded language acquisition that learns to map natural language statements to their referents in a physical environment. For example, given an image, LSP can map the statement “blue mug on the table” to the set of image segments showing blue mugs on tables. LSP learns physical representations for both categorical (“blue,” “mug”) and relational (“on”) language, and also learns to compose these representations to produce the referents of entire statements. We further introduce a weakly supervised training procedure that estimates LSP’s parameters using annotated referents for entire statements, without annotated referents for individual words or the parse structure of the statement. We perform experiments on two applications: scene understanding and geographical question answering. We find that LSP outperforms existing, less expressive models that cannot represent relational language. We further find that weakly supervised training is competitive with fully supervised training while requiring significantly less annotation effort.

scholarly journals PIQA: Reasoning about Physical Commonsense in Natural Language

2020 ◽  
Vol 34 (05) ◽  
pp. 7432-7439
Author(s):  
Yonatan Bisk ◽  
Rowan Zellers ◽  
Ronan Le bras ◽  
Jianfeng Gao ◽  
Yejin Choi
Keyword(s):  
Natural Language ◽  
Question Answering ◽  
Natural Language Understanding ◽  
Physical World ◽  
Future Research ◽  
Physical Interaction ◽  
Commonsense Reasoning ◽  
Cotton Swab ◽  
Language Understanding ◽  
Abstract Domains

To apply eyeshadow without a brush, should I use a cotton swab or a toothpick? Questions requiring this kind of physical commonsense pose a challenge to today's natural language understanding systems. While recent pretrained models (such as BERT) have made progress on question answering over more abstract domains – such as news articles and encyclopedia entries, where text is plentiful – in more physical domains, text is inherently limited due to reporting bias. Can AI systems learn to reliably answer physical commonsense questions without experiencing the physical world?In this paper, we introduce the task of physical commonsense reasoning and a corresponding benchmark dataset Physical Interaction: Question Answering or PIQA. Though humans find the dataset easy (95% accuracy), large pretrained models struggle (∼75%). We provide analysis about the dimensions of knowledge that existing models lack, which offers significant opportunities for future research.

scholarly journals Grounded Language Learning: Where Robotics and NLP Meet

2018 ◽  
Author(s):  
Cynthia Matuszek
Keyword(s):  
Language Acquisition ◽  
Natural Language ◽  
Language Learning ◽  
Physical World ◽  
Research Area ◽  
Embodied Agents ◽  
Future Directions ◽  
Recent Advances ◽  
The World ◽  
And Control

Grounded language acquisition is concerned with learning the meaning of language as it applies to the physical world. As robots become more capable and ubiquitous, there is an increasing need for non-specialists to interact with and control them, and natural language is an intuitive, flexible, and customizable mechanism for such communication. At the same time, physically embodied agents offer a way to learn to understand natural language in the context of the world to which it refers. This paper gives an overview of the research area, selected recent advances, and some future directions and challenges that remain.

Infant-Directed Speech Effects in Adult Artificial Grammar Learning

2019 ◽  
Author(s):  
Krisztina Sára Lukics ◽  
Ágnes Lukács
Keyword(s):  
Language Acquisition ◽  
Natural Language ◽  
First Language ◽  
Higher Learning ◽  
Artificial Grammar ◽  
Combined Effect ◽  
First Language Acquisition ◽  
Relative Contribution ◽  
Grammar Learning ◽  
Learning Rates

First language acquisition is facilitated by several characteristics of infant-directed speech, but we know little about their relative contribution to learning different aspects of language. We investigated infant-directed speech effects on the acquisition of a linear artificial grammar in two experiments. We examined the effect of incremental presentation of strings (starting small) and prosody (comparing monotonous, arbitrary and phrase prosody). Presenting shorter strings before longer ones led to higher learning rates compared to random presentation. Prosody marking phrases had a similar effect, yet, prosody without marking syntactic units did not facilitate learning. These studies were the first to test the starting small effect with a linear artificial grammar, and also the first to investigate the combined effect of starting small and prosody. Our results suggest that starting small and prosody facilitate the extraction of regularities from artificial linguistic stimuli, indicating they may play an important role in natural language acquisition.

Differentiating scalar implicature from exclusion inferences in language acquisition

2017 ◽  
Author(s):  
Jess Sullivan ◽  
Kathryn Davidson ◽  
Shirlene Wade ◽  
David Barner
Keyword(s):  
Language Acquisition ◽  
Young Children ◽  
Scalar Implicature ◽  
Scalar Implicatures ◽  
Mutual Exclusivity ◽  
Logical Semantics

When acquiring language, children must not only learn the meanings of words, but also how to interpret them in context. For example, children must learn both the logical semantics of the scalar quantifier some and its pragmatically enriched meaning: ‘some but not all’. Some studies have shown that this “scalar implicature” that some implies ‘some but not all’ poses a challenge even to nine-year-olds, while others find success by age three. We asked whether reports of children’s early successes might be due to the computation of exclusion inferences (like contrast or mutual exclusivity) rather than an ability to compute scalar implicatures. We found that young children (N=214; ages 4;0-7;11) sometimes prefer to compute symmetrical exclusion inferences rather than asymmetric scalar inferences when interpreting quantifiers. This suggests that some apparent successes in computing scalar implicature can actually be explained by less sophisticated exclusion inferences.

A Hybrid Siamese Neural Network for Natural Language Inference in Cyber-Physical Systems

2021 ◽  
Vol 21 (2) ◽  
pp. 1-25
Author(s):  
Pin Ni ◽  
Yuming Li ◽  
Gangmin Li ◽  
Victor Chang
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Short Term Memory ◽  
Physical World ◽  
Heterogeneous Data ◽  
Cyber Physical Systems ◽  
Physical Systems ◽  
Language Data ◽  
Text Language ◽  
Different Sources

Cyber-Physical Systems (CPS), as a multi-dimensional complex system that connects the physical world and the cyber world, has a strong demand for processing large amounts of heterogeneous data. These tasks also include Natural Language Inference (NLI) tasks based on text from different sources. However, the current research on natural language processing in CPS does not involve exploration in this field. Therefore, this study proposes a Siamese Network structure that combines Stacked Residual Long Short-Term Memory (bidirectional) with the Attention mechanism and Capsule Network for the NLI module in CPS, which is used to infer the relationship between text/language data from different sources. This model is mainly used to implement NLI tasks and conduct a detailed evaluation in three main NLI benchmarks as the basic semantic understanding module in CPS. Comparative experiments prove that the proposed method achieves competitive performance, has a certain generalization ability, and can balance the performance and the number of trained parameters.

scholarly journals Composing Questions through Conceptual Authoring

2007 ◽  
Vol 33 (1) ◽  
pp. 105-133 ◽  
Author(s):  
Catalina Hallett ◽  
Donia Scott ◽  
Richard Power
Keyword(s):  
Natural Language ◽  
Question Answering ◽  
Free Text ◽  
Risk Averse ◽  
Proof Of Concept ◽  
Concept System ◽  
Complex Queries ◽  
Extensive Training ◽  
Question Answering Systems ◽  
Medical Histories

This article describes a method for composing fluent and complex natural language questions, while avoiding the standard pitfalls of free text queries. The method, based on Conceptual Authoring, is targeted at question-answering systems where reliability and transparency are critical, and where users cannot be expected to undergo extensive training in question composition. This scenario is found in most corporate domains, especially in applications that are risk-averse. We present a proof-of-concept system we have developed: a question-answering interface to a large repository of medical histories in the area of cancer. We show that the method allows users to successfully and reliably compose complex queries with minimal training.

Sign in / Sign up

Export Citation Format

Share Document