scholarly journals Deep Reinforcement Learning for Procedural Content Generation of 3D Virtual Environments

2020 ◽  
Vol 20 (5) ◽  
Author(s):  
Christian E. López ◽  
James Cunningham ◽  
Omar Ashour ◽  
Conrad S. Tucker
Keyword(s):  
Reinforcement Learning ◽  
Virtual Environments ◽  
A Priori ◽  
Three Dimensional ◽  
Generative Models ◽  
Primary Objective ◽  
Training Data ◽  
Procedural Content Generation ◽  
3D Virtual Environments ◽  
Content Generation

Abstract This work presents a deep reinforcement learning (DRL) approach for procedural content generation (PCG) to automatically generate three-dimensional (3D) virtual environments that users can interact with. The primary objective of PCG methods is to algorithmically generate new content in order to improve user experience. Researchers have started exploring the use of machine learning (ML) methods to generate content. However, these approaches frequently implement supervised ML algorithms that require initial datasets to train their generative models. In contrast, RL algorithms do not require training data to be collected a priori since they take advantage of simulation to train their models. Considering the advantages of RL algorithms, this work presents a method that generates new 3D virtual environments by training an RL agent using a 3D simulation platform. This work extends the authors’ previous work and presents the results of a case study that supports the capability of the proposed method to generate new 3D virtual environments. The ability to automatically generate new content has the potential to maintain users’ engagement in a wide variety of applications such as virtual reality applications for education and training, and engineering conceptual design.

Reinforcement Learning Content Generation for Virtual Reality Applications

2019 ◽  
Author(s):  
Christian E. Lopez ◽  
Omar Ashour ◽  
Conrad S. Tucker
Keyword(s):  
Virtual Reality ◽  
Reinforcement Learning ◽  
Design Space ◽  
Generative Models ◽  
Primary Objective ◽  
Large Datasets ◽  
Procedural Content Generation ◽  
Learning Content ◽  
Pcg Method ◽  
Content Generation

Abstract This work presents a Procedural Content Generation (PCG) method based on a Neural Network Reinforcement Learning (RL) approach that generates new environments for Virtual Reality (VR) learning applications. The primary objective of PCG methods is to algorithmically generate new content (e.g., environments, levels) in order to improve user experience. Researchers have started exploring the integration of Machine Learning (ML) algorithms into their PCG methods. These ML approaches help explore the design space and generate new content more efficiently. The capability to provide users with new content has great potential for learning applications. However, these ML algorithms require large datasets to train their generative models. In contrast, RL based methods take advantage of simulation to train their models. Moreover, even though VR has become an emerging technology to engage users, there have been few studies that explore PCG for learning purposes and fewer in the context of VR. Considering these limitations, this work presents a method that generates new VR environments by training an RL agent using a simulation platform. This PCG method has the potential to maintain users’ engagement over time by presenting them with new environments in VR learning applications.

Usability and Affordances for Teaching and Learning in Second Life

2012 ◽  
pp. 1307-1322
Author(s):  
Trevor Barker
Keyword(s):  
Virtual Environments ◽  
Small Groups ◽  
Second Life ◽  
Teaching And Learning ◽  
Empirical Studies ◽  
Three Dimensional ◽  
Task Completion ◽  
Science Students ◽  
The University ◽  
3D Virtual Environments

This chapter presents a summary of research undertaken at the University of Hertfordshire into the usability and affordances of three-dimensional (3D) virtual environments (VE) used in teaching and learning. Our earlier experimental work identified important variables related to individual differences and how these affected task completion, learning, and attitude to the environment. More recently the results of these laboratory-based empirical studies have been applied to teaching and learning in the Second Life virtual world. The results of two studies are presented with undergraduate Computer Science students. In the first study the affordances of the Second Life environment for project group working and teaching was evaluated. In the second study small groups of learners developed real world games and modified these for play in Second Life. Based on experiences from these studies, a set of recommendations related to the use of 3D virtual environments in teaching and learning is presented.

Multi-Context Generation in Virtual Reality Environments Using Deep Reinforcement Learning

2020 ◽  
Author(s):  
James Cunningham ◽  
Christian Lopez ◽  
Omar Ashour ◽  
Conrad S. Tucker
Keyword(s):  
Virtual Reality ◽  
Reinforcement Learning ◽  
Virtual Environments ◽  
Probability Distributions ◽  
Automatic Generation ◽  
Grocery Store ◽  
Training Data ◽  
Learning Approaches ◽  
Common Concept

Abstract In this work, a Deep Reinforcement Learning (RL) approach is proposed for Procedural Content Generation (PCG) that seeks to automate the generation of multiple related virtual reality (VR) environments for enhanced personalized learning. This allows for the user to be exposed to multiple virtual scenarios that demonstrate a consistent theme, which is especially valuable in an educational context. RL approaches to PCG offer the advantage of not requiring training data, as opposed to other PCG approaches that employ supervised learning approaches. This work advances the state of the art in RL-based PCG by demonstrating the ability to generate a diversity of contexts in order to teach the same underlying concept. A case study is presented that demonstrates the feasibility of the proposed RL-based PCG method using examples of probability distributions in both manufacturing facility and grocery store virtual environments. The method demonstrated in this paper has the potential to enable the automatic generation of a variety of virtual environments that are connected by a common concept or theme.

scholarly journals Toward Diverse Text Generation with Inverse Reinforcement Learning

2018 ◽  
Author(s):  
Zhan Shi ◽  
Xinchi Chen ◽  
Xipeng Qiu ◽  
Xuanjing Huang
Keyword(s):  
Reinforcement Learning ◽  
Generative Models ◽  
Training Data ◽  
Great Success ◽  
Text Generation ◽  
Inverse Reinforcement Learning ◽  
Reward Function ◽  
Total Reward ◽  
Policy Gradient ◽  
Adversarial Models

Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suffer from the problems of reward sparsity and mode collapse. In order to address these two problems, in this paper, we employ inverse reinforcement learning (IRL) for text generation. Specifically, the IRL framework learns a reward function on training data, and then an optimal policy to maximum the expected total reward. Similar to the adversarial models, the reward and policy function in IRL are optimized alternately. Our method has two advantages: (1) the reward function can produce more dense reward signals. (2) the generation policy, trained by ``entropy regularized'' policy gradient, encourages to generate more diversified texts. Experiment results demonstrate that our proposed method can generate higher quality texts than the previous methods.

scholarly journals Opposition-Based Reinforcement Learning

2006 ◽  
Vol 10 (4) ◽  
pp. 578-585 ◽  
Author(s):  
Hamid R. Tizhoosh ◽  
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
A Priori ◽  
Optimal Solution ◽  
Machine Intelligence ◽  
Training Data ◽  
Superior Performance ◽  
Major Drawback ◽  
Opposition Based Learning ◽  
Dimensional State Space

Reinforcement learning is a machine intelligence scheme for learning in highly dynamic, probabilistic environments. By interaction with the environment, reinforcement agents learn optimal control policies, especially in the absence of a priori knowledge and/or a sufficiently large amount of training data. Despite its advantages, however, reinforcement learning suffers from a major drawback - high calculation cost because convergence to an optimal solution usually requires that all states be visited frequently to ensure that policy is reliable. This is not always possible, however, due to the complex, high-dimensional state space in many applications. This paper introduces opposition-based reinforcement learning, inspired by opposition-based learning, to speed up convergence. Considering opposite actions simultaneously enables individual states to be updated more than once shortening exploration and expediting convergence. Three versions of Q-learning algorithm will be given as examples. Experimental results for the grid world problem of different sizes demonstrate the superior performance of the proposed approach.

scholarly journals Adversarial Reinforcement Learning for Procedural Content Generation

2021 ◽  
Author(s):  
Linus Gisslen ◽  
Andy Eakins ◽  
Camilo Gordillo ◽  
Joakim Bergdahl ◽  
Konrad Tollmar
Keyword(s):  
Reinforcement Learning ◽  
Procedural Content Generation ◽  
Content Generation

scholarly journals Evaluating students’ perception of a three-dimensional virtual world learning environment

2013 ◽  
pp. 323-333
Keyword(s):  
Learning Environment ◽  
Virtual Environments ◽  
Virtual Environment ◽  
Teaching And Learning ◽  
Three Dimensional ◽  
Virtual Classroom ◽  
Virtual Learning Environment ◽  
Perceived Usefulness ◽  
Learning Activities ◽  
3D Virtual Environments

Three-dimensional virtual environments have gained wide popularity due to improvement in graphic rendering technology and networking infrastructure. Many education institutions have been trying to leverage the potential of 3D virtual environments in their application in education. In this research, we aim to evaluate the students’ perception of virtual environments in teaching and learning activities. We set up a virtual classroom, where a short presentation was delivered to students through virtual projectors in Second Life, the most widely adopted 3D virtual environment. The students filled in a questionnaire after the class. We found that the students gave a statistically higher evaluation to 3D virtual environments in terms of satisfaction and enjoyment, while comparable scores between 3D and traditional learning environment in terms of concentration, perceived usefulness, and learning and understanding were obtained. Our results show that virtual learning environment is of great potential in e-learning. Some recommendations in using the virtual environment for learning activities are given.

The Influence of Intelligent Characters on Users' Navigation Through a Three-Dimensional Virtual Environment

2010 ◽  
Vol 19 (3) ◽  
pp. 253-264 ◽  
Author(s):  
Anderson Souza ◽  
de Araújo ◽  
Luís Alfredo Vidal ◽  
de Carvalho ◽  
Rosa Maria E Moreira ◽  
...  
Keyword(s):  
Standard Model ◽  
Virtual Environments ◽  
Virtual Environment ◽  
Three Dimensional ◽  
Physical Appearance ◽  
Art Gallery ◽  
Virtual Characters ◽  
3D Virtual Environments ◽  
Standard Character

Future interfaces will increasingly explore three-dimensional (3D) scenes that will have 3D virtual characters interacting with users. Currently, the characters' influence on users' navigation is not well known. The aim of our research is to study the potential of virtual characters in influencing users' decisions in 3D virtual environments. We implemented an art gallery where human-like characters, with some level of intelligence, can move about and communicate, trying to influence the user's path. When the users enter the gallery, they choose the exhibitions they would like to visit. After that, the characters can assume two positions: “helpful,” which reinforces the previous users' choices, and another one that has a “spirit of contradiction,” suggesting navigation options that are different from those indicated by the users. Five groups of users tested the environment. In some cases, the user could customize the character's physical appearance. In others, a standard model was used. In contrast to the expected results, the experiment indicated that the standard character had more influence on the users' choices than the customized character.

Sign in / Sign up

Export Citation Format

Share Document