scholarly journals Application of Reinforcement Learning to a Robotic Drinking Assistant

Robotics ◽  
2019 ◽  
Vol 9 (1) ◽  
pp. 1 ◽  
Author(s):  
Tejas Kumar Shastha ◽  
Maria Kyrarini ◽  
Axel Gräser
Keyword(s):  
Reinforcement Learning ◽  
Tactile Feedback ◽  
Human Robot Interaction ◽  
Assistive Robotics ◽  
Training Procedure ◽  
Assistive Robots ◽  
Assistive Robot ◽  
Markov Decision ◽  
Degree Of Acceptance ◽  
High Degree

Meal assistant robots form a very important part of the assistive robotics sector since self-feeding is a priority activity of daily living (ADL) for people suffering from physical disabilities like tetraplegia. A quick survey of the current trends in this domain reveals that, while tremendous progress has been made in the development of assistive robots for the feeding of solid foods, the task of feeding liquids from a cup remains largely underdeveloped. Therefore, this paper describes an assistive robot that focuses specifically on the feeding of liquids from a cup using tactile feedback through force sensors with direct human–robot interaction (HRI). The main focus of this paper is the application of reinforcement learning (RL) to learn what the best robotic actions are, based on the force applied by the user. A model of the application environment is developed based on the Markov decision process and a software training procedure is designed for quick development and testing. Five of the commonly used RL algorithms are investigated, with the intention of finding the best fit for training, and the system is tested in an experimental study. The preliminary results show a high degree of acceptance by the participants. Feedback from the users indicates that the assistive robot functions intuitively and effectively.

DESIGNING INTELLIGENT SOCIALLY ASSISTIVE ROBOTS AS EFFECTIVE TOOLS IN COGNITIVE INTERVENTIONS

2011 ◽  
Vol 08 (01) ◽  
pp. 103-126 ◽  
Author(s):  
JEANIE CHAN ◽  
GOLDIE NEJAT ◽  
JINGCONG CHEN
Keyword(s):  
Cognitive Stimulation ◽  
Human Robot Interaction ◽  
Control Architecture ◽  
Cognitive Interventions ◽  
Training Interventions ◽  
Assistive Robots ◽  
Socially Assistive Robots ◽  
Assistive Robot ◽  
Term Basis ◽  
Socially Assistive Robot

Recently, there has been a growing body of research that supports the effectiveness of using non-pharmacological cognitive and social training interventions to reduce the decline of or improve brain functioning in individuals suffering from cognitive impairments. However, implementing and sustaining such interventions on a long-term basis is difficult as they require considerable resources and people, and can be very time-consuming for healthcare staff. Our research focuses on making these interventions more accessible to healthcare professionals through the aid of robotic assistants. The objective of our work is to develop an intelligent socially assistive robot with abilities to recognize and identify human affective intent to determine its own appropriate emotion-based behavior while engaging in assistive interactions with people. In this paper, we present the design of a novel human-robot interaction (HRI) control architecture that allows the robot to provide social and cognitive stimulation in person-centered cognitive interventions. Namely, the novel control architecture is designed to allow a robot to act as a social motivator by encouraging, congratulating and assisting a person during the course of a cognitively stimulating activity. Preliminary experiments validate the effectiveness of the control architecture in providing assistive interactions during a HRI-based person-directed activity.

scholarly journals Effects of Assistive Robot Behavior on Impressions of Patient Psychological Attributes: Vignette-Based Human-Robot Interaction Study

10.2196/13729 ◽  
2019 ◽  
Vol 21 (6) ◽  
pp. e13729 ◽  
Author(s):  
Meia Chita-Tegmark ◽  
Janet M Ackerman ◽  
Matthias Scheutz
Keyword(s):  
Health Care ◽  
Health Care Providers ◽  
Health Management ◽  
Human Robot Interaction ◽  
Care Providers ◽  
Patient Centered ◽  
Human Relationships ◽  
Assistive Robots ◽  
Assistive Robot ◽  
Health Care Context

Background As robots are increasingly designed for health management applications, it is critical to not only consider the effects robots will have on patients but also consider a patient’s wider social network, including the patient’s caregivers and health care providers, among others. Objective In this paper we investigated how people evaluate robots that provide care and how they form impressions of the patient the robot cares for, based on how the robot represents the patient. Methods We have used a vignette-based study, showing participants hypothetical scenarios describing behaviors of assistive robots (patient-centered or task-centered) and measured their influence on people’s evaluations of the robot itself (emotional intelligence [EI], trustworthiness, and acceptability) as well as people’s perceptions of the patient for whom the robot provides care. Results We found that for scenarios describing a robot that acts in a patient-centered manner, the robot will not only be perceived as having higher EI (P=.003) but will also cause people to form more positive impressions of the patient that the robot cares for (P<.001). We replicated and expanded these results to other domains such as dieting, learning, and job training. Conclusions These results imply that robots could be used to enhance human-human relationships in the health care context and beyond.

scholarly journals Development and Preliminary Investigation of a Semiautonomous Socially Assistive Robot (SAR) Designed to Elicit Communication, Motor Skills, Emotion, and Visual Regard (Engagement) from Young Children with Complex Cerebral Palsy: A Pilot Comparative Trial

2019 ◽  
Vol 2019 ◽  
pp. 1-14
Author(s):  
Cecilia Clark ◽  
Levin Sliker ◽  
Jim Sandstrum ◽  
Brian Burne ◽  
Victoria Haggett ◽  
...  
Keyword(s):  
Cerebral Palsy ◽  
Preliminary Investigation ◽  
Comparative Trial ◽  
Assistive Technologies ◽  
Assistive Robotics ◽  
Assistive Robots ◽  
Assistive Robot ◽  
Play Interactions ◽  
Socially Assistive Robot ◽  
Positive Engagement

Through play, typically developing children manipulate objects and interact with peers to establish and develop physical, cognitive, language, and social skills. However, children with complex disabilities and/or developmental delays have limited play experiences, thus compromising the quality of play and acquisition of skills. Assistive technologies have been developed to increase opportunities and level of interaction for children with disabilities to facilitate learning and development. One type of technology, Socially Assistive Robotics, is designed to assist the human user through social interaction while creating measurable growth in learning and rehabilitation. The investigators in this study designed, developed, and validated a semiautonomous Socially Assistive Robot to compare with a switch-adapted toy to determine robot effectiveness in quantity of, changes in, and differences in engagement. After interacting with both systems for three sessions each, five of the eight subjects showed a greater level of positive engagement with the robot than the switch-adapted toy, while the remaining three subjects showed slightly higher positive engagement with the toy. The preliminary results of the study suggest that Socially Assistive Robots specifically designed for children with complex cerebral palsy should be further researched and utilized to enrich play interactions and skill development for this population.

scholarly journals Effects of Assistive Robot Behavior on Impressions of Patient Psychological Attributes: Vignette-Based Human-Robot Interaction Study (Preprint)

2019 ◽  
Author(s):  
Meia Chita-Tegmark ◽  
Janet M Ackerman ◽  
Matthias Scheutz
Keyword(s):  
Health Care ◽  
Health Care Providers ◽  
Health Management ◽  
Human Robot Interaction ◽  
Care Providers ◽  
Patient Centered ◽  
Human Relationships ◽  
Assistive Robots ◽  
Assistive Robot ◽  
Health Care Context

BACKGROUND As robots are increasingly designed for health management applications, it is critical to not only consider the effects robots will have on patients but also consider a patient’s wider social network, including the patient’s caregivers and health care providers, among others. OBJECTIVE In this paper we investigated how people evaluate robots that provide care and how they form impressions of the patient the robot cares for, based on how the robot represents the patient. METHODS We have used a vignette-based study, showing participants hypothetical scenarios describing behaviors of assistive robots (patient-centered or task-centered) and measured their influence on people’s evaluations of the robot itself (emotional intelligence [EI], trustworthiness, and acceptability) as well as people’s perceptions of the patient for whom the robot provides care. RESULTS We found that for scenarios describing a robot that acts in a patient-centered manner, the robot will not only be perceived as having higher EI (P=.003) but will also cause people to form more positive impressions of the patient that the robot cares for (P<.001). We replicated and expanded these results to other domains such as dieting, learning, and job training. CONCLUSIONS These results imply that robots could be used to enhance human-human relationships in the health care context and beyond.

scholarly journals Are Preferences Useful for Better Assistance?

2021 ◽  
Vol 10 (4) ◽  
pp. 1-19
Author(s):  
Gerard Canal ◽  
Carme Torras ◽  
Guillem Alenyà
Keyword(s):  
User Preferences ◽  
Human Robot Interaction ◽  
Assistive Robotics ◽  
User Evaluation ◽  
Robot Interaction ◽  
Assistive Robots ◽  
Robot Behavior

Assistive Robots have an inherent need of adapting to the user they are assisting. This is crucial for the correct development of the task, user safety, and comfort. However, adaptation can be performed in several manners. We believe user preferences are key to this adaptation. In this article, we evaluate the use of preferences for Physically Assistive Robotics tasks in a Human-Robot Interaction user evaluation. Three assistive tasks have been implemented consisting of assisted feeding, shoe-fitting, and jacket dressing, where the robot performs each task in a different manner based on user preferences. We assess the ability of the users to determine which execution of the task used their chosen preferences (if any). The obtained results show that most of the users were able to successfully guess the cases where their preferences were used even when they had not seen the task before. We also observe that their satisfaction with the task increases when the chosen preferences are employed. Finally, we also analyze the user’s opinions regarding assistive tasks and preferences, showing promising expectations as to the benefits of adapting the robot behavior to the user through preferences.

Europe leads the way in assistive robots for the elderly

2017 ◽  
Vol 44 (3) ◽  
pp. 253-258 ◽  
Author(s):  
Robert Bogue
Keyword(s):  
Emerging Market ◽  
Healthcare Providers ◽  
Human Robot Interaction ◽  
Ageing Population ◽  
European Research ◽  
Research Projects ◽  
Content Type ◽  
Assistive Robots ◽  
Stage Of Development ◽  
Assistive Robot

Purpose This paper aims to provide details of European research projects and product developments involving robots that can assist the ageing population. Design/methodology/approach Following an introduction, the role of assistive robots and research into the nature of the human–robot interaction are considered. The paper then discusses a selection of European research projects and a number of companies producing or developing assistive robots. Finally, brief conclusions are drawn. Findings In recognition of the fact that the growing, ageing population has needs that place an unsustainable burden on carers and healthcare providers, Europe is investing heavily in assistive robots. Many European Union-funded, collaborative projects have been conducted and several continue today which build on the extensive body of earlier research. Significant progress is being made, and assistive robot research has moved on from purely technological developments to field trials involving real people in realistic environments. Several products exist or are at an advanced stage of development and have often benefited or arisen from these projects. Europe is in a very strong position to capitalise on this emerging market opportunity. Originality/value This provides a detailed insight into European assistive robot development activities.

A Non-Intrusive Wearable Bio-Sensor Based Assistive Robotic System for Human Mental and Physical Intervention

2017 ◽  
Author(s):  
Kyle Lansing ◽  
Wei Yu ◽  
Biswanath Samanta
Keyword(s):  
Mobile Robot ◽  
Human Subjects ◽  
Vital Role ◽  
Human Robot Interaction ◽  
Assistive Robotics ◽  
Activity Levels ◽  
Concept System ◽  
Assistive Robots ◽  
Conductivity Sensor ◽  
Human Stress

Assistive robotics and technologies are going to play a vital role in our society. These platforms can support a level of human-robot interaction that is more meaningful, accommodating, and effective. This is especially true in the realms of medicine and rehabilitation, although assistive robots have a wider range of applications. In this work, using a non-intrusive wearable bio-sensor, a PC, and a mobile robot a novel proof of concept system has been developed that can detect human mental and physical states and intervene to promote mental and physical wellbeing. This study has utilized a skin-conductivity sensor to monitor changes in galvanic skin response (GSR) due to the presence of stress or anxiety along with a three-axis accelerometer to detect changes in physical activity levels. Two data processing algorithms have been developed to identify the mental and physical states by employing trend analysis techniques. By programming the system to obtain a baseline reading for individual subjects and comparing subsequent sensor values sustained changes in GSR levels due to stress can be detected. Similarly, by utilizing arrays and monitoring changes in accelerometer readings pattern changes associated with different physical activities can be detected. In addition, behaviors and motions aimed at alleviating human mental stress and physical inactivity have been developed by employing distraction and reminder intervention methods using a mobile robot. Experiments have been conducted on human subjects to evaluate the proposed robotic system’s capability to identify mental and physical states and intervene to improve their situation through participant responses. Based on the responses, a mean rating of 4.41 and 4.83 out of 5 has been given for the system’s ability to recognize human stress and physical state respectively. Additionally, participants have reported a mean of 30.3% reduction of stress and a mean of 23.3% increase in positive mood following the system’s intervention behavior.

scholarly journals Inverse reinforcement learning in contextual MDPs

2021 ◽  
Author(s):  
Stav Belogolovsky ◽  
Philip Korsunsky ◽  
Shie Mannor ◽  
Chen Tessler ◽  
Tom Zahavy
Keyword(s):  
Reinforcement Learning ◽  
Optimization Problem ◽  
Decision Processes ◽  
Inverse Reinforcement Learning ◽  
Convex Optimization Problem ◽  
Reward Function ◽  
Dynamic Treatment Regime ◽  
Markov Decision ◽  
Dynamic Treatment ◽  
Recorded Data

AbstractWe consider the task of Inverse Reinforcement Learning in Contextual Markov Decision Processes (MDPs). In this setting, contexts, which define the reward and transition kernel, are sampled from a distribution. In addition, although the reward is a function of the context, it is not provided to the agent. Instead, the agent observes demonstrations from an optimal policy. The goal is to learn the reward mapping, such that the agent will act optimally even when encountering previously unseen contexts, also known as zero-shot transfer. We formulate this problem as a non-differential convex optimization problem and propose a novel algorithm to compute its subgradients. Based on this scheme, we analyze several methods both theoretically, where we compare the sample complexity and scalability, and empirically. Most importantly, we show both theoretically and empirically that our algorithms perform zero-shot transfer (generalize to new and unseen contexts). Specifically, we present empirical experiments in a dynamic treatment regime, where the goal is to learn a reward function which explains the behavior of expert physicians based on recorded data of them treating patients diagnosed with sepsis.

scholarly journals Optimal Policies for Quantum Markov Decision Processes

2021 ◽  
Author(s):  
Ming-Sheng Ying ◽  
Yuan Feng ◽  
Sheng-Gang Ying
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Quantum Systems ◽  
Sequential Decision Making ◽  
Mathematical Framework ◽  
Sequential Decision ◽  
Learning Techniques ◽  
Optimal Policies ◽  
Markov Decision ◽  
Programming Algorithms

AbstractMarkov decision process (MDP) offers a general framework for modelling sequential decision making where outcomes are random. In particular, it serves as a mathematical framework for reinforcement learning. This paper introduces an extension of MDP, namely quantum MDP (qMDP), that can serve as a mathematical model of decision making about quantum systems. We develop dynamic programming algorithms for policy evaluation and finding optimal policies for qMDPs in the case of finite-horizon. The results obtained in this paper provide some useful mathematical tools for reinforcement learning techniques applied to the quantum world.

scholarly journals Can a humanoid social robot stimulate the interactivity of cognitively impaired elderly? A thorough study based on computer vision methods

2021 ◽  
Author(s):  
Gauri Tulsulkar ◽  
Nidhi Mishra ◽  
Nadia Magnenat Thalmann ◽  
Hwee Er Lim ◽  
Mei Ping Lee ◽  
...  
Keyword(s):  
Computer Vision ◽  
Psychosocial Support ◽  
Well Being ◽  
The Elderly ◽  
Social Robots ◽  
Assistive Robotics ◽  
Cognitively Impaired ◽  
Assistive Robots ◽  
Institutional Review ◽  
Human Need

AbstractSocial Assistive Robotics is increasingly being used in care settings to provide psychosocial support and interventions for the elderly with cognitive impairments. Most of these social robots have provided timely stimuli to the elderly at home and in care centres, including keeping them active and boosting their mood. However, previous investigations have registered shortcomings in these robots, particularly in their ability to satisfy an essential human need: the need for companionship. Reports show that the elderly tend to lose interests in these social robots after the initial excitement as the novelty wears out and the monotonous familiarity becomes all too familiar. This paper presents our research facilitating conversations between a social humanoid robot, Nadine, and cognitively impaired elderly at a nursing home. We analysed the effectiveness of human–humanoid interactions between our robot and 14 elderly over 29 sessions. We used both objective tools (based on computer vision methods) and subjective tools (based on observational scales) to evaluate the recorded videos. Our findings showed that our subjects engaged positively with Nadine, suggesting that their interaction with the robot could improve their well-being by compensating for some of their emotional, cognitive, and psychosocial deficiencies. We detected emotions associated with cognitively impaired elderly during these interactions. This study could help understand the expectations of the elderly and the current limitations of Social Assistive Robots. Our research is aligned with all the ethical recommendations by the NTU Institutional Review Board.

Sign in / Sign up

Export Citation Format

Share Document