A Natural User Interface to Drive a Robotic Fish

2015 ◽  
Author(s):  
Mauro De Bellis ◽  
Paul Phamduy ◽  
Maurizio Porfiri
Keyword(s):  
User Interface ◽  
User Interfaces ◽  
Ad Hoc ◽  
Robotic Fish ◽  
Microsoft Kinect ◽  
User Interest ◽  
Natural User Interface ◽  
Informal Science Learning ◽  
Interactive Control ◽  
Natural User Interfaces

Interactive control modes for robotic fish based informal science learning activities have been shown to increase user interest in STEM careers. This study explores the use of natural user interfaces to engage users in an interactive activity and excite them about the possibility of a robotics career. In this work, we propose a novel natural user interface platform for enhancing participant interaction by controlling a robotic fish in a set of tasks. Specifically, we develop and characterize a new platform, which utilizes a Microsoft Kinect and an ad-hoc communication protocol. Preliminary studies are conducted to assess the usability of the platform.

Hand’s poses recognition as a mean of communication within natural user interfaces

2012 ◽  
Vol 60 (2) ◽  
pp. 331-336 ◽  
Author(s):  
A. Wojciechowski
Keyword(s):  
User Interface ◽  
User Interfaces ◽  
Graphical User Interfaces ◽  
Hand Tracking ◽  
Command Line ◽  
Medical Systems ◽  
Recognition Method ◽  
Natural User Interface ◽  
Natural Approach ◽  
Natural User Interfaces

Abstract. Natural user interface (NUI) is a successor of command line interfaces (CLI) and graphical user interfaces (GUI) so well known to computer users. A new natural approach is based on extensive human behaviors tracking, where hand tracking and gesture recognition seem to play the main roles in communication. The presented paper reviews common approaches to discussed hand features tracking and provides a very effective proposal of the contour based hand’s poses recognition method which can be straightforwardly used for a hand-based natural user interface. Its possible usage varies from medical systems interaction, through games up to impaired people communication support.

scholarly journals Enfoque liviano para reconocimiento de gestos manuales híbridos con cámaras de profundidad

10.52278/2415 ◽  
2020 ◽  
Author(s):  
Diego Gabriel Alonso
Keyword(s):  
User Interface ◽  
Human Computer Interaction ◽  
Activity Recognition ◽  
Human Activity ◽  
Human Activity Recognition ◽  
Microsoft Kinect ◽  
First Person ◽  
Leap Motion ◽  
Motion Controller ◽  
Natural User Interface

En los últimos años, en combinación con los avances tecnológicos han surgido nuevos paradigmas de interacción con el usuario. Esto ha motivado a la industria a la creación de dispositivos de Interfaz Natural de Usuario (NUI, del inglés Natural User Interface) cada vez más potentes y accesibles. En particular, las cámaras de profundidad han alcanzado grandes niveles de adopción por parte de los usuarios. Entre estos dispositivos se destacan la Microsoft Kinect, la Intel RealSense y el Leap Motion Controller. Este tipo de dispositivos facilitan la adquisición de datos en el Reconocimiento de Actividades Humanas (HAR, del inglés Human Activity Recognition). HAR es un área que tiene por objetivo la identificación automática, dentro de secuencias de imágenes, de actividades realizadas por seres humanos. Entre los diferentes tipos de actividades humanas se encuentran los gestos manuales, es decir, aquellos realizados con las manos. Los gestos manuales pueden ser estáticos o dinámicos, según si presentan movimiento en las secuencias de imágenes. El reconocimiento de gestos manuales permite a los desarrolladores de sistemas de Interacción Humano-Computadora (HCI, del inglés Human-Computer Interaction) crear experiencias e interacciones más inmersivas, naturales e intuitivas. Sin embargo, esta tarea no resulta sencilla. Es por ello que, en la academia se ha abordado esta problemática con el uso de técnicas de aprendizaje de máquina. Tras el análisis del estado del arte actual, se ha identificado que la gran mayoría de los enfoques propuestos no contemplan el reconocimiento de los gestos estáticos y los dinámicos en forma simultánea (enfoques híbridos). Es decir, los enfoques están destinados a reconocer un solo tipo de gestos. Además, dado el contexto de sistemas HCI reales debe tenerse en cuenta también el costo computacional y el consumo de recursos de estos enfoques, con lo cual los enfoques deberían ser livianos. Por otra parte, casi la totalidad de los enfoques presentes en el estado del arte abordan la problemática ubicando las cámaras frente a los usuarios (perspectiva de segunda persona) y no desde la perspectiva de primera persona (FPV, del inglés First-Person View), en la que el usuario posee un dispositivo colocado sobre sí mismo. Esto puede asociarse con que recién en los últimos años han surgido dispositivos relativamente ergonómicos (pequeños, de peso ligero) que permitan considerar una perspectiva FPV viable. En este contexto, en la presente tesis se propone un enfoque liviano para el reconocimiento de gestos híbridos con cámaras de profundidad teniendo en cuenta la perspectiva FPV. El enfoque propuesto consta de 3 grandes componentes. En primer lugar, el de Adquisición de Datos, en el cual se define el dispositivo a utilizar y se recopilan las imágenes y la información de profundidad que es normalizada al rango de valores de 0 a 255 (escala de los canales RGB). En segundo lugar, el de Preprocesamiento, el cual tiene por objetivo hacer que dos secuencias de imágenes con variaciones temporales sean comparables. Para ello, se aplican técnicas de remuestreo y reducción de resolución. Además, en este componente se computa el flujo óptico determinado por las secuencias de imágenes a color que se poseen. En particular, se utiliza el flujo óptico como un nuevo canal de información dadas sus ventajas en lo que respecta a un análisis espacio-temporal de los videos. En tercer lugar, con las secuencias muestreadas y con la información de flujo óptico, se procede al componente Modelo de Aprendizaje Profundo, donde se aplican técnicas de aprendizaje profundo que permiten abordar las etapas de extracción de características y de clasificación. Particularmente, se propone una arquitectura de red convolucional densamente conectada con soporte multi-modal. Cabe destacar que, la fusión de las modalidades no es en etapa temprana ni tardía sino dentro del mismo modelo. De esta manera, se obtiene un modelo end-to-end que obtiene beneficios de los canales de información en forma separada y también conjunta. Los experimentos realizados han mostrado resultados muy alentadores (alcanzando un 90% de exactitud) indicando que la elección de este tipo de arquitecturas permite obtener una gran eficiencia de parámetros así como también de tiempos de predicción. Cabe resaltar que, las pruebas son realizadas sobre un conjunto de datos relevante del área. En base a ello, se analiza el desempeño de la presente propuesta en relación a diferentes escenarios como con variación de iluminación o movimiento de cámara, diferentes tipos de gestos, sensibilidad o sesgo por personas, entre otros.

scholarly journals Natural User Interfaces in Volume Visualisation Using Microsoft Kinect

2013 ◽  
pp. 11-19 ◽  
Author(s):  
Anastassia Angelopoulou ◽  
José García-Rodríguez ◽  
Alexandra Psarrou ◽  
Markos Mentzelopoulos ◽  
Bharat Reddy ◽  
...  
Keyword(s):  
User Interfaces ◽  
Microsoft Kinect ◽  
Natural User Interfaces ◽  
Volume Visualisation

scholarly journals Web GIS in practice X: a Microsoft Kinect natural user interface for Google Earth navigation

2011 ◽  
Vol 10 (1) ◽  
pp. 45 ◽  
Author(s):  
Maged N Kamel Boulos ◽  
Bryan J Blanchard ◽  
Cory Walker ◽  
Julio Montero ◽  
Aalap Tripathy ◽  
...  
Keyword(s):  
User Interface ◽  
Google Earth ◽  
Microsoft Kinect ◽  
Web Gis ◽  
Natural User Interface

Kinect Applications in Healthcare

2019 ◽  
pp. 391-402
Author(s):  
Roanna Lun ◽  
Wenbing Zhao
Keyword(s):  
User Interface ◽  
Research And Development ◽  
Motion Detection ◽  
Microphone Array ◽  
Low Cost ◽  
Microsoft Kinect ◽  
Depth Camera ◽  
Detection Capability ◽  
Natural User Interface ◽  
Research Issues

Microsoft Kinect is one of the most popular inexpensive gadgets released in recent years. Kinect is equipped with a color camera, a depth camera, and a microphone array. The device allows users to interact with a computer via a natural user interface in terms of gestures or voice commands. The authors believe that the research and development on using Kinect technology in healthcare will gain more momentum. The demand of Kinect-based applications is high, due to Kinect's low cost and portability, and its accurate and robust motion detection capability. In this chapter, the authors survey the current applications of using the Kinect technology in healthcare. Furthermore, they outline a number of open research issues that could overcome the limitations of the current Kinect technology.

scholarly journals CONTROL OF A DRONE WITH BODY GESTURES

2021 ◽  
Vol 1 ◽  
pp. 761-770
Author(s):  
Nicolas Gio ◽  
Ross Brisco ◽  
Tijana Vuletic
Keyword(s):  
User Interface ◽  
Graphical User Interface ◽  
Functional System ◽  
Recognition Algorithm ◽  
Microsoft Kinect ◽  
Civil Aviation ◽  
Video Capture ◽  
Natural User Interface ◽  
Military Applications ◽  
And Control

AbstractDrones are becoming more popular within military applications and civil aviation by hobbyists and business. Achieving a natural Human-Drone Interaction (HDI) would enable unskilled drone pilots to take part in the flying of these devices and more generally easy the use of drones. The research within this paper focuses on the design and development of a Natural User Interface (NUI) allowing a user to pilot a drone with body gestures. A Microsoft Kinect was used to capture the user’s body information which was processed by a motion recognition algorithm and converted into commands for the drone. The implementation of a Graphical User Interface (GUI) gives feedback to the user. Visual feedback from the drone’s onboard camera is provided on a screen and an interactive menu controlled by body gestures and allowing the choice of functionalities such as photo and video capture or take-off and landing has been implemented. This research resulted in an efficient and functional system, more instinctive, natural, immersive and fun than piloting using a physical controller, including innovative aspects such as the implementation of additional functionalities to the drone's piloting and control of the flight speed.

scholarly journals Spud 1.0: generalising and automating the user interfaces of scientific computer models

2008 ◽  
Vol 1 (1) ◽  
pp. 125-146 ◽  
Author(s):  
D. A. Ham ◽  
P. E. Farrell ◽  
G. J. Gorman ◽  
J. R. Maddison ◽  
C. R. Wilson ◽  
...  
Keyword(s):  
User Interface ◽  
User Interfaces ◽  
Ad Hoc ◽  
Computer Models ◽  
Scientific Models ◽  
Development Cost ◽  
Wide Range ◽  
Model Independent ◽  
Scientific Computer ◽  
User Friendly

Abstract. The interfaces by which users specify the scenarios to be simulated by scientific computer models are frequently primitive, under-documented and ad-hoc text files which make using the model in question difficult and error-prone and significantly increase the development cost of the model. In this paper, we present a model-independent system, Spud, which formalises the specification of model input formats in terms of formal grammars. This is combined with an automated graphical user interface which guides users to create valid model inputs based on the grammar provided, and a generic options reading module which minimises the development cost of adding model options. Together, this provides a user friendly, well documented, self validating user interface which is applicable to a wide range of scientific models and which minimises the developer input required to maintain and extend the model interface.

scholarly journals Development and evaluation of two posture-tracking user interfaces for occupational health care

2018 ◽  
Vol 10 (6) ◽  
pp. 168781401876948 ◽  
Author(s):  
Sergio Valdivia ◽  
Robin Blanco ◽  
Alvaro Uribe-Quevedo ◽  
Lina Penuela ◽  
David Rojas ◽  
...  
Keyword(s):  
User Interface ◽  
Occupational Health ◽  
User Interfaces ◽  
Mobile Application ◽  
Inertial Measurement Unit ◽  
Spinal Column ◽  
Microsoft Kinect ◽  
Measurement Unit ◽  
Inertial Measurement ◽  
Posture Tracking

The spinal column requires special care through exercises focused on muscle strengthening, flexibility, and mobility to minimize the risk of developing musculoskeletal disorders that may affect the quality of life. Guidelines for spinal column exercises are commonly presented through printed and multimedia guides accompanied with demonstrations performed by a physiotherapist, occupational health expert, or physical fitness trainer. However, existing guides lack interaction and oral explanations may not always be clear to the user, leading to decreased engagement and motivation to start, continue, or complete an exercise program. In this article, we present two interactive and engaging posture-tracking user interfaces intended to promote proper spinal column exercise form. One user interface employs a wooden manikin with an integrated inertial measurement unit to provide a tangible user interaction. The other user interface presents a mobile application that provides instructions and explanations about the exercises. Both user interfaces allow recording key postures during the exercise for reference and feedback. We compared the usability of the interfaces through a series of flexion and extension exercises, monitored with an inertial measuring unit worn around the torso, and a Microsoft Kinect V2 vision-based sensor. Although no significant differences between the manikin user interface and the mobile application were found in terms of usability, the inertial measurement unit provided more accurate and reliable data in comparison to the Microsoft Kinect V2 as a result of body occlusions in front of the sensor caused during the torso flexion. Although both user interfaces provide different experiences and performed well, we believe that a combination of both will improve user engagement and motivation, while providing a more accurate motion profile.

Sign in / Sign up

Export Citation Format

Share Document