scholarly journals Enhanced Action Recognition Using Multiple Stream Deep Learning with Optical Flow and Weighted Sum

Sensors ◽  
2020 ◽  
Vol 20 (14) ◽  
pp. 3894
Author(s):  
Hyunwoo Kim ◽  
Seokmok Park ◽  
Hyeokjin Park ◽  
Joonki Paik
Keyword(s):  
Optical Flow ◽  
Action Recognition ◽  
Three Dimensional ◽  
Video Frame ◽  
Weighted Sum ◽  
Flow Noise ◽  
Stream Structure ◽  
Flow Information ◽  
Score Fusion ◽  
Improving Accuracy

Various action recognition approaches have recently been proposed with the aid of three-dimensional (3D) convolution and a multiple stream structure. However, existing methods are sensitive to background and optical flow noise, which prevents from learning the main object in a video frame. Furthermore, they cannot reflect the accuracy of each stream in the process of combining multiple streams. In this paper, we present a novel action recognition method that improves the existing method using optical flow and a multi-stream structure. The proposed method consists of two parts: (i) optical flow enhancement process using image segmentation and (ii) score fusion process by applying weighted sum of the accuracy. The enhancement process can help the network to efficiently analyze the flow information of the main object in the optical flow frame, thereby improving accuracy. A different accuracy of each stream can be reflected to the fused score while using the proposed score fusion method. We achieved an accuracy of 98.2% on UCF-101 and 82.4% on HMDB-51. The proposed method outperformed many state-of-the-art methods without changing the network structure and it is expected to be easily applied to other networks.

scholarly journals Single-shot detection of 8 unique monochrome fringe patterns representing 4 distinct directions via multispectral fringe projection profilometry

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Parsa Omidi ◽  
Mohamadreza Najiminaini ◽  
Mamadou Diop ◽  
Jeffrey J. L. Carson
Keyword(s):  
Spatial Resolution ◽  
Fringe Pattern ◽  
Three Dimensional ◽  
Fringe Projection ◽  
Video Frame ◽  
Single Shot ◽  
Shot Detection ◽  
Fringe Patterns ◽  
Fringe Projection Profilometry ◽  
Multispectral Camera

AbstractSpatial resolution in three-dimensional fringe projection profilometry is determined in large part by the number and spacing of fringes projected onto an object. Due to the intensity-based nature of fringe projection profilometry, fringe patterns must be generated in succession, which is time-consuming. As a result, the surface features of highly dynamic objects are difficult to measure. Here, we introduce multispectral fringe projection profilometry, a novel method that utilizes multispectral illumination to project a multispectral fringe pattern onto an object combined with a multispectral camera to detect the deformation of the fringe patterns due to the object. The multispectral camera enables the detection of 8 unique monochrome fringe patterns representing 4 distinct directions in a single snapshot. Furthermore, for each direction, the camera detects two π-phase shifted fringe patterns. Each pair of fringe patterns can be differenced to generate a differential fringe pattern that corrects for illumination offsets and mitigates the effects of glare from highly reflective surfaces. The new multispectral method solves many practical problems related to conventional fringe projection profilometry and doubles the effective spatial resolution. The method is suitable for high-quality fast 3D profilometry at video frame rates.

Weighted Voting of Multi-Stream Convolutional Neural Networks for Video-Based Action Recognition using Optical Flow Rhythms

2021 ◽  
pp. 103112
Author(s):  
André Souza Brito ◽  
Marcelo Bernardes Vieira ◽  
Saulo Moraes Villela ◽  
Hemerson Tacon ◽  
Hugo Lima Chaves ◽  
...  
Keyword(s):  
Neural Networks ◽  
Optical Flow ◽  
Convolutional Neural Networks ◽  
Action Recognition ◽  
Weighted Voting

scholarly journals Fast, Efficient Fragment-Based Coordinate Generation for Open Babel

2019 ◽  
Author(s):  
Naruki Yoshikawa ◽  
Geoffrey Hutchison
Keyword(s):  
Molecular Modeling ◽  
Structure Prediction ◽  
Distance Geometry ◽  
Three Dimensional ◽  
Property Prediction ◽  
Crystallographic Databases ◽  
Fragment Library ◽  
Wide Range ◽  
Computational Property ◽  
Improving Accuracy

<div>Rapidly predicting an accurate three dimensional geometry of a molecule is a crucial task in cheminformatics and a range of molecular modeling. Fast, accurate, and open implementation of structure prediction is necessary for reproducible cheminformatics research. We introduce fragment-based coordinate generation for Open Babel, a widely accepted open source toolkit for cheminformatics. The new implementation significant improves speed and stereochemical accuracy, while retaining or improving accuracy of bond lengths, bond angles, and dihedral torsions. We first separate an input molecule into fragments by cutting at rotatable bonds. Coordinates of fragments are set according to the fragment library, which is prepared from open crystallographic databases. Since coordinates of multiple atoms are decided at once, coordinate prediction is accelerated over the previous rules-based implementation or the widely-used distance geometry methods in RDKit. This new implementation will be beneficial for a wide range of applications, including computational property prediction in polymers, molecular materials and drug design.</div>

Monocular Camera Three Dimensional Reconstruction Based on Optical Flow Feedback

2015 ◽  
Vol 35 (5) ◽  
pp. 0515001 ◽  
Author(s):  
李秀智 Li Xiuzhi ◽  
杨爱林 Yang Ailin ◽  
秦宝岭 Qin Baoling ◽  
贾松敏 Jia Songmin ◽  
邱欢 Qiu Huan
Keyword(s):  
Optical Flow ◽  
Three Dimensional ◽  
Three Dimensional Reconstruction ◽  
Monocular Camera ◽  
Dimensional Reconstruction

Continuous Bidirectional Optical Flow for Video Frame Sequence Interpolation

2019 ◽  
Author(s):  
Donghao Gu ◽  
ZhaoJing Wen ◽  
Wenxue Cui ◽  
Rui Wang ◽  
Feng Jiang ◽  
...  
Keyword(s):  
Optical Flow ◽  
Video Frame ◽  
Frame Sequence
Sign in / Sign up

Export Citation Format

Share Document