An Arbitrary Point Tracking using Multi-scale Refined Optical Flow

2007 ◽  
Author(s):  
Jaekwang Lee ◽  
Chang-Joon Park ◽  
In-Ho Lee
Keyword(s):  
Optical Flow ◽  
Arbitrary Point ◽  
Multi Scale ◽  
Point Tracking

scholarly journals A Fast 4K Video Frame Interpolation Using a Multi-Scale Optical Flow Reconstruction Network

Symmetry ◽  
2019 ◽  
Vol 11 (10) ◽  
pp. 1251 ◽  
Author(s):  
Ahn ◽  
Jeong ◽  
Kim ◽  
Kwon ◽  
Yoo
Keyword(s):  
High Resolution ◽  
Optical Flow ◽  
State Of The Art ◽  
Interpolation Method ◽  
Video Frame ◽  
Frame Interpolation ◽  
Multi Scale ◽  
Reconstruction Scheme ◽  
Flow Reconstruction

Recently, video frame interpolation research developed with a convolutional neural network has shown remarkable results. However, these methods demand huge amounts of memory and run time for high-resolution videos, and are unable to process a 4K frame in a single pass. In this paper, we propose a fast 4K video frame interpolation method, based upon a multi-scale optical flow reconstruction scheme. The proposed method predicts low resolution bi-directional optical flow, and reconstructs it into high resolution. We also proposed consistency and multi-scale smoothness loss to enhance the quality of the predicted optical flow. Furthermore, we use adversarial loss to make the interpolated frame more seamless and natural. We demonstrated that the proposed method outperforms the existing state-of-the-art methods in quantitative evaluation, while it runs up to 4.39× faster than those methods for 4K videos.

Analysis of Human Accident Video Using Optical Flow With Physically-Based Constraints

2013 ◽  
Author(s):  
Christine Zwart ◽  
David Frakes ◽  
William Singhose
Keyword(s):  
Optical Flow ◽  
Poor Quality ◽  
Logical Approach ◽  
Automated Tracking ◽  
Motion Data ◽  
Point Tracking ◽  
Points Of Interest ◽  
Physically Based ◽  
Poor Lighting

Videos that capture accidents are usually of poor quality: they are likely to be taken from a bad angle, with poor lighting, and contain occluded points. However, the motion data contained in such videos can be very valuable for understanding and preventing accidents. To extract the motion of a body from a video: 1) the points of interest must be identified and 2) point tracking from frame-to-frame must be accomplished. Accordingly, one logical approach is to focus on automated tracking, while allowing a human to identify important points of interest [1].

scholarly journals Deep Optical Flow Estimation Via Multi-Scale Correspondence Structure Learning

2017 ◽  
Author(s):  
Shanshan Zhao ◽  
Xi Li ◽  
Omar El Farouk Bourahla
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Structure Learning ◽  
Feature Space ◽  
Image Correlation ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Multi Scale ◽  
Benchmark Datasets ◽  
End To End

As an important and challenging problem in computer vision, learning based optical flow estimation aims to discover the intrinsic correspondence structure between two adjacent video frames through statistical learning. Therefore, a key issue to solve in this area is how to effectively model the multi-scale correspondence structure properties in an adaptive end-to-end learning fashion. Motivated by this observation, we propose an end-to-end multi-scale correspondence structure learning (MSCSL) approach for optical flow estimation. In principle, the proposed MSCSL approach is capable of effectively capturing the multi-scale inter-image-correlation correspondence structures within a multi-level feature space from deep learning. Moreover, the proposed MSCSL approach builds a spatial Conv-GRU neural network model to adaptively model the intrinsic dependency relationships among these multi-scale correspondence structures. Finally, the above procedures for correspondence structure learning and multi-scale dependency modeling are implemented in a unified end-to-end deep learning framework. Experimental results on several benchmark datasets demonstrate the effectiveness of the proposed approach.

scholarly journals RAFT: Recurrent All-Pairs Field Transforms for Optical Flow (Extended Abstract)

2021 ◽  
Author(s):  
Zachary Teed ◽  
Jia Deng
Keyword(s):  
Flow Field ◽  
Optical Flow ◽  
Network Architecture ◽  
High Efficiency ◽  
State Of The Art ◽  
Deep Network ◽  
Multi Scale ◽  
Art Performance

We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes. RAFT achieves state-of-the-art performance on the KITTI and Sintel datasets. In addition, RAFT has strong cross-dataset generalization as well as high efficiency in inference time, training speed, and parameter count.

scholarly journals Multi-scale joint feature network for micro-expression recognition

2021 ◽  
Author(s):  
Xinyu Li ◽  
Guangshun Wei ◽  
Jie Wang ◽  
Yuanfeng Zhou
Keyword(s):  
Optical Flow ◽  
Expression Recognition ◽  
Facial Motion ◽  
Multi Scale ◽  
Flow Image ◽  
Wide Range ◽  
Recognition Ability ◽  
Benchmark Datasets ◽  
Casme Ii ◽  
Micro Expression

AbstractMicro-expression recognition is a substantive cross-study of psychology and computer science, and it has a wide range of applications (e.g., psychological and clinical diagnosis, emotional analysis, criminal investigation, etc.). However, the subtle and diverse changes in facial muscles make it difficult for existing methods to extract effective features, which limits the improvement of micro-expression recognition accuracy. Therefore, we propose a multi-scale joint feature network based on optical flow images for micro-expression recognition. First, we generate an optical flow image that reflects subtle facial motion information. The optical flow image is then fed into the multi-scale joint network for feature extraction and classification. The proposed joint feature module (JFM) integrates features from different layers, which is beneficial for the capture of micro-expression features with different amplitudes. To improve the recognition ability of the model, we also adopt a strategy for fusing the feature prediction results of the three JFMs with the backbone network. Our experimental results show that our method is superior to state-of-the-art methods on three benchmark datasets (SMIC, CASME II, and SAMM) and a combined dataset (3DB).

Sign in / Sign up

Export Citation Format

Share Document