Benchmarking The Imbalanced Behavior of Deep Learning Based Optical Flow Estimators

2019 ◽  
Author(s):  
Stefano Savian ◽  
Mehdi Elahi ◽  
Tammam Tillo
Keyword(s):  
Deep Learning ◽  
Optical Flow

scholarly journals Deep Learning-Based Congestion Detection at Urban Intersections

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2052
Author(s):  
Xinghai Yang ◽  
Fengjiao Wang ◽  
Zhiquan Bai ◽  
Feifei Xun ◽  
Yulin Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Traffic Congestion ◽  
Detection Algorithm ◽  
Input Image ◽  
Vehicle Speed ◽  
Position Information ◽  
Traffic State ◽  
State Discrimination ◽  
Discrimination Method

In this paper, a deep learning-based traffic state discrimination method is proposed to detect traffic congestion at urban intersections. The detection algorithm includes two parts, global speed detection and a traffic state discrimination algorithm. Firstly, the region of interest (ROI) is selected as the road intersection from the input image of the You Only Look Once (YOLO) v3 object detection algorithm for vehicle target detection. The Lucas-Kanade (LK) optical flow method is employed to calculate the vehicle speed. Then, the corresponding intersection state can be obtained based on the vehicle speed and the discrimination algorithm. The detection of the vehicle takes the position information obtained by YOLOv3 as the input of the LK optical flow algorithm and forms an optical flow vector to complete the vehicle speed detection. Experimental results show that the detection algorithm can detect the vehicle speed and traffic state discrimination method can judge the traffic state accurately, which has a strong anti-interference ability and meets the practical application requirements.

Fusion of rain radar images and wind forecasts in adeep learning model applied to rain nowcasting

2021 ◽  
Author(s):  
Anastase Charantonis ◽  
Vincent Bouget ◽  
Dominique Béréziat ◽  
Julien Brajard ◽  
Arthur Filoche
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Weather Forecast ◽  
Forecast Model ◽  
Radar Data ◽  
Learning Model ◽  
Rainfall Forecasting ◽  
Radar Images ◽  
Deep Learning Model ◽  
Rain Radar

<p>Short or mid-term rainfall forecasting is a major task with several environmental applications such as agricultural management or flood risks monitoring. Existing data-driven approaches, especially deep learning models, have shown significant skill at this task, using only rainfall radar images as inputs. In order to determine whether using other meteorological parameters such as wind would improve forecasts, we trained a deep learning model on a fusion of rainfall radar images and wind velocity produced by a weather forecast model. The network was compared to a similar architecture trained only on radar data, to a basic persistence model and to an approach based on optical flow. Our network outperforms by 8% the F1-score calculated for the optical flow on moderate and higher rain events for forecasts at a horizon time of 30 minutes. Furthermore, it outperforms by 7% the same architecture trained using only rainfall radar images. Merging rain and wind data has also proven to stabilize the training process and enabled significant improvement especially on the difficult-to-predict high precipitation rainfalls. These results can also be found in Bouget, V., Béréziat, D., Brajard, J., Charantonis, A., & Filoche, A. (2020). Fusion of rain radar images and wind forecasts in a deep learning model applied to rain nowcasting. arXiv preprint arXiv:2012.05015</p>

scholarly journals Superresolution Reconstruction of Video Based on Efficient Subpixel Convolutional Neural Network for Urban Computing

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Jie Shen ◽  
Mengxi Xu ◽  
Xinyu Du ◽  
Yunbo Xiong
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Optical Flow ◽  
Video Surveillance ◽  
Reconstruction Method ◽  
Motion Information ◽  
Learning Networks ◽  
Urban Computing ◽  
Deep Convolution Network

Video surveillance is an important data source of urban computing and intelligence. The low resolution of many existing video surveillance devices affects the efficiency of urban computing and intelligence. Therefore, improving the resolution of video surveillance is one of the important tasks of urban computing and intelligence. In this paper, the resolution of video is improved by superresolution reconstruction based on a learning method. Different from the superresolution reconstruction of static images, the superresolution reconstruction of video is characterized by the application of motion information. However, there are few studies in this area so far. Aimed at fully exploring motion information to improve the superresolution of video, this paper proposes a superresolution reconstruction method based on an efficient subpixel convolutional neural network, where the optical flow is introduced in the deep learning network. Fusing the optical flow features between successive frames can compensate for information in frames and generate high-quality superresolution results. In addition, in order to improve the superresolution, a superpixel convolution layer is added after the deep convolution network. Finally, experimental evaluations demonstrate the satisfying performance of our method compared with previous methods and other deep learning networks; our method is more efficient.

scholarly journals Automated Video Behavior Recognition of Pigs Using Two-Stream Convolutional Networks

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1085
Author(s):  
Kaifeng Zhang ◽  
Dan Li ◽  
Jiayun Huang ◽  
Yifei Chen
Keyword(s):  
Feature Extraction ◽  
Deep Learning ◽  
Optical Flow ◽  
Network Models ◽  
Well Being ◽  
Motion Information ◽  
Behavior Recognition ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Effective Manner

The detection of pig behavior helps detect abnormal conditions such as diseases and dangerous movements in a timely and effective manner, which plays an important role in ensuring the health and well-being of pigs. Monitoring pig behavior by staff is time consuming, subjective, and impractical. Therefore, there is an urgent need to implement methods for identifying pig behavior automatically. In recent years, deep learning has been gradually applied to the study of pig behavior recognition. Existing studies judge the behavior of the pig only based on the posture of the pig in a still image frame, without considering the motion information of the behavior. However, optical flow can well reflect the motion information. Thus, this study took image frames and optical flow from videos as two-stream input objects to fully extract the temporal and spatial behavioral characteristics. Two-stream convolutional network models based on deep learning were proposed, including inflated 3D convnet (I3D) and temporal segment networks (TSN) whose feature extraction network is Residual Network (ResNet) or the Inception architecture (e.g., Inception with Batch Normalization (BN-Inception), InceptionV3, InceptionV4, or InceptionResNetV2) to achieve pig behavior recognition. A standard pig video behavior dataset that included 1000 videos of feeding, lying, walking, scratching and mounting from five kinds of different behavioral actions of pigs under natural conditions was created. The dataset was used to train and test the proposed models, and a series of comparative experiments were conducted. The experimental results showed that the TSN model whose feature extraction network was ResNet101 was able to recognize pig feeding, lying, walking, scratching, and mounting behaviors with a higher average of 98.99%, and the average recognition time of each video was 0.3163 s. The TSN model (ResNet101) is superior to the other models in solving the task of pig behavior recognition.

scholarly journals Deep Optical Flow Estimation Via Multi-Scale Correspondence Structure Learning

2017 ◽  
Author(s):  
Shanshan Zhao ◽  
Xi Li ◽  
Omar El Farouk Bourahla
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Structure Learning ◽  
Feature Space ◽  
Image Correlation ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Multi Scale ◽  
Benchmark Datasets ◽  
End To End

As an important and challenging problem in computer vision, learning based optical flow estimation aims to discover the intrinsic correspondence structure between two adjacent video frames through statistical learning. Therefore, a key issue to solve in this area is how to effectively model the multi-scale correspondence structure properties in an adaptive end-to-end learning fashion. Motivated by this observation, we propose an end-to-end multi-scale correspondence structure learning (MSCSL) approach for optical flow estimation. In principle, the proposed MSCSL approach is capable of effectively capturing the multi-scale inter-image-correlation correspondence structures within a multi-level feature space from deep learning. Moreover, the proposed MSCSL approach builds a spatial Conv-GRU neural network model to adaptively model the intrinsic dependency relationships among these multi-scale correspondence structures. Finally, the above procedures for correspondence structure learning and multi-scale dependency modeling are implemented in a unified end-to-end deep learning framework. Experimental results on several benchmark datasets demonstrate the effectiveness of the proposed approach.

scholarly journals Multi‐task deep learning with optical flow features for self‐driving cars

2020 ◽  
Vol 14 (13) ◽  
pp. 1845-1854
Author(s):  
Yuan Hu ◽  
Hubert P. H. Shum ◽  
Edmond S. L. Ho
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Flow Features ◽  
Self Driving Cars ◽  
Optical Flow Features

scholarly journals Optical Flow Estimation and Denoising of Video Images Based on Deep Learning Models

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 144122-144135
Author(s):  
Ang Li ◽  
Baoyu Zheng ◽  
Lei Li ◽  
Chen Zhang
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Learning Models ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Video Images
Sign in / Sign up

Export Citation Format

Share Document