Multimodal data fusion for object recognition

2019 ◽  
Author(s):  
Vladimir A. Knyaz
Keyword(s):  
Object Recognition ◽  
Data Fusion ◽  
Multimodal Data ◽  
Multimodal Data Fusion

Online Learning for Multimodal Data Fusion With Application to Object Recognition

2018 ◽  
Vol 65 (9) ◽  
pp. 1259-1263 ◽  
Author(s):  
Shahin Shahrampour ◽  
Mohammad Noshad ◽  
Jie Ding ◽  
Vahid Tarokh
Keyword(s):  
Online Learning ◽  
Object Recognition ◽  
Data Fusion ◽  
Multimodal Data ◽  
Multimodal Data Fusion

Multimodal data fusion framework enhanced robot-assisted minimally invasive surgery

2021 ◽  
pp. 014233122098435
Author(s):  
Wen Qi ◽  
Hang Su ◽  
Ke Fan ◽  
Ziyang Chen ◽  
Jiehao Li ◽  
...  
Keyword(s):  
Minimally Invasive Surgery ◽  
Minimally Invasive ◽  
Data Fusion ◽  
Invasive Surgery ◽  
Robot Control ◽  
Surgical Robot ◽  
Robot Assisted ◽  
Multimodal Data ◽  
Multimodal Data Fusion ◽  
Fusion Framework

The generous application of robot-assisted minimally invasive surgery (RAMIS) promotes human-machine interaction (HMI). Identifying various behaviors of doctors can enhance the RAMIS procedure for the redundant robot. It bridges intelligent robot control and activity recognition strategies in the operating room, including hand gestures and human activities. In this paper, to enhance identification in a dynamic situation, we propose a multimodal data fusion framework to provide multiple information for accuracy enhancement. Firstly, a multi-sensors based hardware structure is designed to capture varied data from various devices, including depth camera and smartphone. Furthermore, in different surgical tasks, the robot control mechanism can shift automatically. The experimental results evaluate the efficiency of developing the multimodal framework for RAMIS by comparing it with a single sensor system. Implementing the KUKA LWR4+ in a surgical robot environment indicates that the surgical robot systems can work with medical staff in the future.

scholarly journals Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation

2020 ◽  
Vol 64 ◽  
pp. 149-187 ◽  
Author(s):  
Yu-Dong Zhang ◽  
Zhengchao Dong ◽  
Shui-Hua Wang ◽  
Xiang Yu ◽  
Xujing Yao ◽  
...  
Keyword(s):  
Data Fusion ◽  
Multimodal Data ◽  
Multimodal Data Fusion

scholarly journals Exploring Multimodal Data Fusion Through Joint Decompositions with Flexible Couplings

2016 ◽  
Vol 64 (18) ◽  
pp. 4830-4844 ◽  
Author(s):  
Rodrigo Cabral Farias ◽  
Jeremy Emile Cohen ◽  
Pierre Comon
Keyword(s):  
Data Fusion ◽  
Multimodal Data ◽  
Multimodal Data Fusion

scholarly journals A Survey on Deep Learning for Multimodal Data Fusion

2020 ◽  
Vol 32 (5) ◽  
pp. 829-864 ◽  
Author(s):  
Jing Gao ◽  
Peng Li ◽  
Zhikui Chen ◽  
Jianing Zhang
Keyword(s):  
Big Data ◽  
Deep Learning ◽  
Data Fusion ◽  
High Volume ◽  
Fusion Method ◽  
Learning Models ◽  
Multimodal Data ◽  
Fusion Methods ◽  
Multimodal Data Fusion ◽  
Future Topics

With the wide deployments of heterogeneous networks, huge amounts of data with characteristics of high volume, high variety, high velocity, and high veracity are generated. These data, referred to multimodal big data, contain abundant intermodality and cross-modality information and pose vast challenges on traditional data fusion methods. In this review, we present some pioneering deep learning models to fuse these multimodal big data. With the increasing exploration of the multimodal big data, there are still some challenges to be addressed. Thus, this review presents a survey on deep learning for multimodal data fusion to provide readers, regardless of their original community, with the fundamentals of multimodal deep learning fusion method and to motivate new multimodal data fusion techniques of deep learning. Specifically, representative architectures that are widely used are summarized as fundamental to the understanding of multimodal deep learning. Then the current pioneering multimodal data fusion deep learning models are summarized. Finally, some challenges and future topics of multimodal data fusion deep learning models are described.

Sign in / Sign up

Export Citation Format

Share Document