Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs

Miguel Lázaro-Gredilla; Dianhuan Lin; J. Swaroop Guntupalli; Dileep George

doi:10.1126/scirobotics.aav3150

Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs

Science Robotics ◽

10.1126/scirobotics.aav3150 ◽

2019 ◽

Vol 4 (26) ◽

pp. eaav3150 ◽

Cited By ~ 10

Author(s):

Miguel Lázaro-Gredilla ◽

Dianhuan Lin ◽

J. Swaroop Guntupalli ◽

Dileep George

Keyword(s):

Working Memory ◽

Computer Architecture ◽

Arm Movement ◽

Physical World ◽

Instruction Set ◽

Computational Framework ◽

Perception System ◽

Image Pairs ◽

High Level ◽

Transfer Tasks

Humans can infer concepts from image pairs and apply those in the physical world in a completely different setting, enabling tasks like IKEA assembly from diagrams. If robots could represent and infer high-level concepts, then it would notably improve their ability to understand our intent and to transfer tasks between different environments. To that end, we introduce a computational framework that replicates aspects of human concept learning. Concepts are represented as programs on a computer architecture consisting of a visual perception system, working memory, and action controller. The instruction set of this cognitive computer has commands for parsing a visual scene, directing gaze and attention, imagining new objects, manipulating the contents of a visual working memory, and controlling arm movement. Inferring a concept corresponds to inducing a program that can transform the input to the output. Some concepts require the use of imagination and recursion. Previously learned concepts simplify the learning of subsequent, more elaborate concepts and create a hierarchy of abstractions. We demonstrate how a robot can use these abstractions to interpret novel concepts presented to it as schematic images and then apply those concepts in very different situations. By bringing cognitive science ideas on mental imagery, perceptual symbols, embodied cognition, and deictic mechanisms into the realm of machine learning, our work brings us closer to the goal of building robots that have interpretable representations and common sense.

Download Full-text

Pattern of Near Transfer Effects Following Working Memory Training With a Dual N-Back Task

Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie) ◽

10.1027/1618-3169/a000370 ◽

2017 ◽

Vol 64 (4) ◽

pp. 240-252 ◽

Cited By ~ 12

Author(s):

Anna Soveri ◽

Eric P. A. Karlsson ◽

Otto Waris ◽

Petra Grönholm-Nyman ◽

Matti Laine

Keyword(s):

Working Memory ◽

Controlled Trial ◽

Composite Score ◽

Memory Training ◽

Interference Control ◽

Transfer Effects ◽

Near Transfer ◽

Randomized Controlled ◽

Transfer Tasks ◽

Back Task

Abstract. In a randomized controlled trial, we investigated the pattern of near transfer effects of working memory (WM) training with an adaptive auditory-visuospatial dual n-back training task in healthy young adults. The results revealed significant task-specific transfer to an untrained single n-back task, and more general near transfer to a WM updating composite score plus a nearly significant effect on a composite score measuring interference control in WM. No transfer effects were seen on Active or Passive WM composites. The results are discussed in the light of cognitive versus strategy-related overlap between training and transfer tasks.

Download Full-text

Evaluation of the Consistency of Simultaneously Acquired Sentinel-2 and Landsat 8 Imagery on Plastic Covered Greenhouses

Remote Sensing ◽

10.3390/rs12122015 ◽

2020 ◽

Vol 12 (12) ◽

pp. 2015 ◽

Cited By ~ 2

Author(s):

Manuel Ángel Aguilar ◽

Rafael Jiménez-Lao ◽

Abderrahim Nemmaoui ◽

Fernando José Aguilar ◽

Dilek Koc-San ◽

...

Keyword(s):

Vegetation Index ◽

Normalized Difference Vegetation Index ◽

Coefficient Of Determination ◽

Landsat 8 ◽

Surface Reflectance ◽

Remote Sensing Techniques ◽

Image Pairs ◽

Study Sites ◽

High Level ◽

Sentinel 2

Remote sensing techniques based on medium resolution satellite imagery are being widely applied for mapping plastic covered greenhouses (PCG). This article aims at testing the spectral consistency of surface reflectance values of Sentinel-2 MSI (S2 L2A) and Landsat 8 OLI (L8 L2 and the pansharpened and atmospherically corrected product from L1T product; L8 PANSH) data in PCG areas located in Spain, Morocco, Italy and Turkey. The six corresponding bands of S2 and L8, together with the normalized difference vegetation index (NDVI), were generated through an OBIA approach for each PCG study site. The coefficient of determination (r2) and the root mean square error (RMSE) were computed in sixteen cloud-free simultaneously acquired image pairs from the four study sites to evaluate the coherence between the two sensors. It was found that the S2 and L8 correlation (r2 > 0.840, RMSE < 9.917%) was quite good in most bands and NDVI. However, the correlation of the two sensors fluctuated between study sites, showing occasional sun glint effects on PCG roofs related to the sensor orbit and sun position. Moreover, higher surface reflectance discrepancies between L8 L2 and L8 PANSH data, mainly in the visible bands, were always observed in areas with high-level aerosol values derived from the aerosol quality band included in the L8 L2 product (SR aerosol). In this way, the consistency between L8 PANSH and S2 L2A was improved mainly in high-level aerosol areas according to the SR aerosol band.

Download Full-text

One-IPC high-level simulation of microthreaded many-core architectures

The International Journal of High Performance Computing Applications ◽

10.1177/1094342015584495 ◽

2016 ◽

Vol 31 (2) ◽

pp. 152-162 ◽

Cited By ~ 3

Author(s):

Irfan Uddin

Keyword(s):

Design Space Exploration ◽

Instruction Set ◽

Efficient Design ◽

Simulation Framework ◽

Fine Grained ◽

Detailed Simulation ◽

High Level ◽

Many Core ◽

The Cost ◽

Multiple Clusters

The microthreaded many-core architecture is comprised of multiple clusters of fine-grained multi-threaded cores. The management of concurrency is supported in the instruction set architecture of the cores and the computational work in application is asynchronously delegated to different clusters of cores, where the cluster is allocated dynamically. Computer architects are always interested in analyzing the complex interaction amongst the dynamically allocated resources. Generally a detailed simulation with a cycle-accurate simulation of the execution time is used. However, the cycle-accurate simulator for the microthreaded architecture executes at the rate of 100,000 instructions per second, divided over the number of simulated cores. This means that the evaluation of a complex application executing on a contemporary multi-core machine can be very slow. To perform efficient design space exploration we present a co-simulation environment, where the detailed execution of instructions in the pipeline of microthreaded cores and the interactions amongst the hardware components are abstracted. We present the evaluation of the high-level simulation framework against the cycle-accurate simulation framework. The results show that the high-level simulator is faster and less complicated than the cycle-accurate simulator but with the cost of losing accuracy.

Download Full-text

Book review: High-Level Language Computer Architecture edited by Veljko Milutinovic (Computer Science Press, 1989)

ACM SIGARCH Computer Architecture News ◽

10.1145/379126.773540 ◽

1990 ◽

Vol 18 (1) ◽

pp. 120-122

Author(s):

Robert P. Colwell

Keyword(s):

Computer Science ◽

Computer Architecture ◽

High Level Language ◽

High Level

Download Full-text

How high level of anxiety in Panic Disorder can interfere in working memory? A computer simulation and electrophysiological investigation

Journal of Psychiatric Research ◽

10.1016/j.jpsychires.2017.08.021 ◽

2017 ◽

Vol 95 ◽

pp. 238-246 ◽

Cited By ~ 1

Author(s):

Luiza Wanick Di Giorgio Silva ◽

Danielle Aprigio ◽

Jesse Di Giacomo ◽

Mariana Gongora ◽

Henning Budde ◽

...

Keyword(s):

Working Memory ◽

Computer Simulation ◽

Panic Disorder ◽

Electrophysiological Investigation ◽

High Level

Download Full-text

Spontaneous emergence of leadership patterns drives synchronization in complex human networks

Scientific Reports ◽

10.1038/s41598-021-97656-y ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Carmela Calabrese ◽

Maria Lombardi ◽

Erik Bollt ◽

Pietro De Lellis ◽

Benoît G. Bardy ◽

...

Keyword(s):

Motor Coordination ◽

Research Effort ◽

Motor Task ◽

Arm Movement ◽

Physical World ◽

Physical Interaction ◽

Digital World ◽

Spatial Configurations ◽

Human Networks ◽

Synchronization Performance

AbstractSynchronization of human networks is fundamental in many aspects of human endeavour. Recently, much research effort has been spent on analyzing how motor coordination emerges in human groups (from rocking chairs to violin players) and how it is affected by coupling structure and strength. Here we uncover the spontaneous emergence of leadership (based on physical signaling during group interaction) as a crucial factor steering the occurrence of synchronization in complex human networks where individuals perform a joint motor task. In two experiments engaging participants in an arm movement synchronization task, in the physical world as well as in the digital world, we found that specific patterns of leadership emerged and increased synchronization performance. Precisely, three patterns were found, involving a subtle interaction between phase of the motion and amount of influence. Such patterns were independent of the presence or absence of physical interaction, and persisted across manipulated spatial configurations. Our results shed light on the mechanisms that drive coordination and leadership in human groups, and are consequential for the design of interactions with artificial agents, avatars or robots, where social roles can be determinant for a successful interaction.

Download Full-text

Digital Design Ecology to Generate a Speculative Virtual Environment Reimagining New Relativity Laws

10.26686/wgtn.14838126.v1 ◽

2021 ◽

Author(s):

J Rogers ◽

Marc Aurel Schnabel ◽

Tane Moleta

Keyword(s):

Design Methodology ◽

Computational Design ◽

Physical World ◽

Virtual Space ◽

Digital Design ◽

Design Concepts ◽

Built Form ◽

High Level ◽

Physical Laws

This paper presents the trilogy of virtual classifications, the speculative environment, the virtual inhabitant and the virtual built-form. These combine, generating a new realm of design within immersive architectural space, all to be designed relative to each other, this paper focuses on the speculative environment portion. This challenged computational design and representation through atmospheric filters, visible environment boundaries, materiality and audio experience. The speculative environment was generated manipulating the physical laws of the physical world, applied within the virtual space. The outcome provided a new spatial experience of architectural dynamics enhanced by detailed spatial qualities. Design concepts within this paper suggest at what immersive virtual reality can evolve into. Following an interconnective design methodology framework allowed a high level of complexity and richness to shine through the research case study throughout the process and final dissemination stages.

Download Full-text

Digital Design Ecology to Generate a Speculative Virtual Environment Reimagining New Relativity Laws

10.26686/wgtn.14838126 ◽

2021 ◽

Author(s):

J Rogers ◽

Marc Aurel Schnabel ◽

Tane Moleta

Keyword(s):

Design Methodology ◽

Computational Design ◽

Physical World ◽

Virtual Space ◽

Digital Design ◽

Design Concepts ◽

Built Form ◽

High Level ◽

Physical Laws

Download Full-text

How Strong Is the Evidence for the Effectiveness of Working Memory Training?

Cognitive and Working Memory Training ◽

10.1093/oso/9780199974467.003.0004 ◽

2019 ◽

pp. 58-76

Author(s):

Claudia C. von Bastian ◽

Sabrina Guye ◽

Carla De Simoni

Keyword(s):

Working Memory ◽

Cognitive Abilities ◽

Small Sample ◽

Sound Studies ◽

Working Memory Training ◽

Memory Training ◽

Sample Sizes ◽

Small Sample Sizes ◽

Transfer Tasks ◽

Larger Sample

This chapter argues that the question of whether working memory training can induce cognitive plasticity in terms of transfer effects cannot be conclusively answered yet due to persisting methodological issues across the literature. The shortcomings discussed include the lack of theoretically motivated selection of training and transfer tasks, the lack of active control groups, and small sample sizes. These problems call into question the strength of the existing evidence. Indeed, reevaluating published findings with Bayesian inference indicated that only a subset of published studies contributed interpretable evidence. The chapter concludes that the current body of literature cannot conclusively support claims that WM training does or does not improve cognitive abilities and stresses the need for theory-driven, methodologically sound studies with larger sample sizes.

Download Full-text