Robust voice user interface for internet-of-things

Internet of things (IoT) plays significant role in the fourth industrial revolution and attracts an increasing interest due to the rapid development of smart devices. IoT comprises factors of twofold. Firstly, a set of things (i.e., appliances, devices, vehicles, etc.) connected together via network. Secondly, human-device interaction to communicate with these things. Speech is the most natural methodology of interaction that can enrich user experience. In this paper, we propose a novel and effective approach for building customized voice interaction for controlling smart devices in IoT environments (i.e., Smart home). The proposed approach is based on extracting customized tiny decoding graph from a large graph constructed using weighted finite sates transducers. Experimental results showed that tiny decoding graphs are very efficient in terms of computational resources and recognition accuracy in clean and noisy conditions. To emphasize the effectiveness of the proposed approach, the standard Resources Management (RM1) dataset was employed and promising results were achieved when compared with four competitive approaches.

Development of a Full-body OpenSim Musculoskeletal Model Incorporating Head-mounted Virtual Reality Headset

In this study, we developed and validated a full-body musculoskeletal model in OpenSim to estimate muscle and joint forces while performing various motor tasks using a virtual reality (VR) system. We compared the results from our developed full-body musculoskeletal model to those from previous studies by simulating kinematic and kinetic data of participants performing pick-and-place lifting tasks using with and without a physically interactive VR system. Results showed that scaling errors between the two environments are comparable, while the overall errors were consistent with previous studies. Overall, the results from the inverse dynamic simulations showed the promise of our developed OpenSim models in determining potential intervention or prevention strategies to reduce the musculoskeletal injury incidences while simulating human-device interaction tasks.

TEACHING THE ARABIC LANGUAGE TO NON-NATIVE SPEAKERS FROM A DISTANCE BETWEEN REALITY AND EXPECTATIONS

The Arabic language is distinguished by its characteristics that distinguish it from other languages, as it is phonetically and symbolically characterized as well as being distinguished by the number of its letters, which reached twenty-eight letters, and also characterized by the letter “Dhad”. It uses this human device to its fullest and best, and does not neglect one of its functions. The Arabic language has a link with the Islamic religion that will remain until the Day of Resurrection because the one who pledges and cares for it is “Allah“ , the Almighty. Most of the non- native speakers teach the Arabic language in order for them to understand the Islam and Its message. Thus, the Arabic language is greatly concerned ad enjoyed by the non- native leaners . But this desire and demand hindered with difficulties, including the lack of a good teacher, as well as teaching methods, so hope for the development of teaching Arabic to non-native speakers remains possible if we benefit from modern technology in teaching Arabic to non- native leaners. This is what the current research will touch upon.

Multi-Component Design of a Mobile Rehabilitation Application for Remote Monitoring of Patients

This paper a will provide background and research outcomes for a student-led multi-University motion capture design project with the FDA Human Device Interaction Lab. This research project spawned from the FDA’s interest in studying mobile Apps used to collect information about a rehabilitation patient’s function outside of a laboratory setting. This is particularly important from a research perspective as it allows knowledge to be gathered about the reliability, safety, and efficacy of such devices, which can later inform human factors guidelines and regulatory oversight. While there is a wide range of applications for remote monitoring of patient function, this project focuses specifically on wearables used to track rehabilitative progress of individuals suffering from upper or lower limb impairment. These wearables may also be used to assess the efficacy of a new assistive technology intended to restore or augment function. The interdisciplinary student teams applied human-centered design principles to develop and prototype hardware and software components for a rotatable body-worn mobile device holder to remotely track rehabilitation progress. The design process and outcomes are presented along with insights on further development of system components.

The adaptability of assistive mobility devices and the role of the internet of medical things: A literature review (Preprint)

UNSTRUCTURED With the projected upsurge in the percentage of persons with some form of disability, there is a significant increase in the need for assistive mobility devices. However, these mobility aids are hardly effective without their ability to adapt to the user’s needs. This is achieved by improving the confidence of the information used or interaction between the user and his device also referred to as adaptation. In the recent past, there has been little effort to provide literature reviews on the adaptability of assistive mobility devices (AMDs). This paper systematically reviews the recent assistive mobility technologies, over the past decade, according to their adaptation and the role that the Internet of Medical Things (IoMT) has played in the adaptability of these technologies. The information gathered in the study provides awareness of the status of adaptive mobility technology and serves as a source and reference of information to healthcare professionals, and researchers. The paper starts by highlighting recent technologies according to the user system interface (human/device interface), then presents some recent technologies in perception and sensor fusion (autonomous navigation) for adaptability, and finally, IoMT frameworks for AMDs. Some notable limitations are also discussed. The findings of the review reveal that an improvement in the adaptation of assistive mobility systems would require a reduction in the training time and avoidance of cognitive overload. Furthermore, sensor fusion and classification accuracy are critical to achieving real-world testing requirements. Finally, the trade-off between cost and performance needs to be considered in the commercialization of these devices.

The Rise of Interactional Multimodality in Human-Computer Interaction

The field of human-computer interaction (HCI) investigates the intersection between the design of devices and users. From an early focus on interaction modeling based on psychological experiments, the field has since experienced a shift towards the study of how actual humans interact with autonomous devices. The field became conductive to ethnographic, observational and videographic studies of human-device interaction. Conversation-analytic HCI became possible. That said, this new wave of researchers was never truly able to dethrone the psychological common sense of the field. With recent developments in both the technical-sensorial capabilities and outward actuational range of embodied virtual agents, the field of HCI has once again returned to the question of the sequential unfolding of the interaction between users and intelligent agents, and the multimodal interactional repertoire that is deployed throughout. This review will highlight the situational orientation of high-impact research in the field, and relate it to the cotemporaneous development of ethnomethodological and conversation analytic frameworks. Acknowledgments. The article was prepared in the framework of a research grant funded by the Ministry of Science and Higher Education of the Russian Federation (grant ID: 075-15-2020-908). The article was prepared in cooperation with the Sber (ex. – Sberbank’s) Gamification Lab.

Age- and Gender-Related Differences in Speech Alignment Toward Humans and Voice-AI

Speech alignment is where talkers subconsciously adopt the speech and language patterns of their interlocutor. Nowadays, people of all ages are speaking with voice-activated, artificially-intelligent (voice-AI) digital assistants through phones or smart speakers. This study examines participants’ age (older adults, 53–81 years old vs. younger adults, 18–39 years old) and gender (female and male) on degree of speech alignment during shadowing of (female and male) human and voice-AI (Apple’s Siri) productions. Degree of alignment was assessed holistically via a perceptual ratings AXB task by a separate group of listeners. Results reveal that older and younger adults display distinct patterns of alignment based on humanness and gender of the human model talkers: older adults displayed greater alignment toward the female human and device voices, while younger adults aligned to a greater extent toward the male human voice. Additionally, there were other gender-mediated differences observed, all of which interacted with model talker category (voice-AI vs. human) or shadower age category (OA vs. YA). Taken together, these results suggest a complex interplay of social dynamics in alignment, which can inform models of speech production both in human-human and human-device interaction.