Channel Attention Is All You Need for Video Frame Interpolation

Prevailing video frame interpolation techniques rely heavily on optical flow estimation and require additional model complexity and computational cost; it is also susceptible to error propagation in challenging scenarios with large motion and heavy occlusion. To alleviate the limitation, we propose a simple but effective deep neural network for video frame interpolation, which is end-to-end trainable and is free from a motion estimation network component. Our algorithm employs a special feature reshaping operation, referred to as PixelShuffle, with a channel attention, which replaces the optical flow computation module. The main idea behind the design is to distribute the information in a feature map into multiple channels and extract motion information by attending the channels for pixel-level frame synthesis. The model given by this principle turns out to be effective in the presence of challenging motion and occlusion. We construct a comprehensive evaluation benchmark and demonstrate that the proposed approach achieves outstanding performance compared to the existing models with a component for optical flow computation.

Assimilation and processing of observation data obtained by satellite earth sensing for monitoring the current state of heterogeneous objects on the water surface

The study is devoted to the analysis of satellite observations data assimilation to discover the necessary information for developing and verifying mathematical models of hydrodynamics and biological shallowwater kinetics. The use of satellite earth sensing data is taken to enhance information base. The possible use of neural networks with optical flow computation is considered in the study. The objective of the study is to develop a software tool used to identify the initial conditions in mathematical modeling of hydrobilogical shallow-water processes.

Stereo vision-based variational optical flow estimation

Optical flow computation is one of the most important tasks in computer vision. The article deals with a modification of the variational method of the optical flow computation, according to its application in stereo vision. Such approaches are traditionally based on a brightness constancy assumption and a gradient constancy assumption during pixels motion. Smoothness assumption also restricts motion discontinuities, i.e. the smoothness of the vector field of pixel velocity is assumed. It is proposed to extend the functional of the optical flow computation in a similar way by adding a priori known stereo cameras extrinsic parameters and minimize such jointed model of optical flow computation. The article presents a partial differential equations framework in image processing and numerical scheme for its implementation. Performed experimental evaluation demonstrates that the proposed method gives smaller errors than traditional methods of optical flow computation.

Hallucinating Optical Flow Features for Video Classification

Appearance and motion are two key components to depict and characterize the video content. Currently, the two-stream models have achieved state-of-the-art performances on video classification. However, extracting motion information, specifically in the form of optical flow features, is extremely computationally expensive, especially for large-scale video classification. In this paper, we propose a motion hallucination network, namely MoNet, to imagine the optical flow features from the appearance features, with no reliance on the optical flow computation. Specifically, MoNet models the temporal relationships of the appearance features and exploits the contextual relationships of the optical flow features with concurrent connections. Extensive experimental results demonstrate that the proposed MoNet can effectively and efficiently hallucinate the optical flow features, which together with the appearance features consistently improve the video classification performances. Moreover, MoNet can help cutting down almost a half of computational and data-storage burdens for the two-stream video classification. Our code is available at: https://github.com/YongyiTang92/MoNet-Features

A new road tracking method based on heading direction detection

Current research on road tracking is mostly based on the visual perception of road boundaries. In this paper, we propose a novel and general road tracker based on optical flow computation, which can be applied to most of road environments including the case of a lack of lane markings or road boundaries. When the heading direction of the vehicle and the road direction are identical, the focus of expansion (FOE) coincides with the road vanishing point (RVP). This is an important foundation for the subsequent heading direction departure decision. By comparing the relative positions of the estimated FOE and RVP, we can learn the traveling state of the vehicle. The experimental results show that the proposed tracker is a simple and efficient road tracking method.