Scaling simulation-to-real transfer by learning a latent space of robot skills

We present a strategy for simulation-to-real transfer, which builds on recent advances in robot skill decomposition. Rather than focusing on minimizing the simulation–reality gap, we propose a method for increasing the sample efficiency and robustness of existing simulation-to-real approaches which exploits hierarchy and online adaptation. Instead of learning a unique policy for each desired robotic task, we learn a diverse set of skills and their variations, and embed those skill variations in a continuously parameterized space. We then interpolate, search, and plan in this space to find a transferable policy which solves more complex, high-level tasks by combining low-level skills and their variations. In this work, we first characterize the behavior of this learned skill space, by experimenting with several techniques for composing pre-learned latent skills. We then discuss an algorithm which allows our method to perform long-horizon tasks never seen in simulation, by intelligently sequencing short-horizon latent skills. Our algorithm adapts to unseen tasks online by repeatedly choosing new skills from the latent space, using live sensor data and simulation to predict which latent skill will perform best next in the real world. Importantly, our method learns to control a real robot in joint-space to achieve these high-level tasks with little or no on-robot time, despite the fact that the low-level policies may not be perfectly transferable from simulation to real, and that the low-level skills were not trained on any examples of high-level tasks. In addition to our results indicating a lower sample complexity for families of tasks, we believe that our method provides a promising template for combining learning-based methods with proven classical robotics algorithms such as model-predictive control.

Measuring Psychology Students’ Information-Seeking Skills in a Situational Judgment Test Format

Three studies were conducted to develop a test for academic information-seeking skills in psychology students that measures both procedural and declarative aspects of the concept. A skill decomposition breaking down information-seeking into 10 sub skills was used to create a situational judgment test with 22 items. A scoring key was developed based on expert ratings (N = 14). Subsequently, the test was administered to two samples of N = 78 and N = 81 psychology students. Within the first sample, the scale reached an internal consistency (Cronbach’s Alpha) of α = .75. Scale validity was investigated with data from the second sample. High correlations between the scale and two different information search tasks (r = .42 to .64; p < .001) as well as a declarative information literacy test (r = .51; p < .001) were found. The findings are discussed with regard to their implications for research and practice.

Neuro-fuzzy-based skill learning for robots

SUMMARYEndowing robots with the ability of skill learning enables them to be versatile and skillful in performing various tasks. This paper proposes a neuro-fuzzy-based, self-organizing skill-learning framework, which differs from previous work in its capability of decomposing a skill by self-categorizing it into significant stimulus-response units (SRU, a fundamental unit of our skill representation), and self-organizing learned skills into a new skill. The proposed neuro-fuzzy-based, self-organizing skill-learning framework can be realized by skill decomposition and skill synthesis. Skill decomposition aims at representing a skill and acquiring it by SRUs, and is implemented by stages with a five-layer neuro-fuzzy network with supervised learning, resolution control, and reinforcement learning to enable robots to identify a sufficient number of significant SRUs for accomplishing a given task without extraneous actions. Skill synthesis aims at organizing a new skill by sequentially planning learned skills composed of SRUs, and is realized by stages, which establish common SRUs between two similar skills and self-organize a new skill from these common SRUs and additional new SRUs by reinforcement learning. Computer simulations and experiments with a Pioneer 3-DX mobile robot were conducted to validate the self-organizing capability of the proposed skill-learning framework in identifying significant SRUs from task examples and in common SRUs between similar skills and learning new skills from learned skills.