scholarly journals A Framework for Real-Time 3D Freeform Manipulation of Facial Video

2019 ◽  
Vol 9 (21) ◽  
pp. 4707
Author(s):  
Jungsik Park ◽  
Byung-Kuk Seo ◽  
Jong-Il Park
Keyword(s):  
Real Time ◽  
Computational Cost ◽  
Video Frame ◽  
Face Model ◽  
Live Video ◽  
Time Performance ◽  
Face Region ◽  
3D Manipulation ◽  
Comparison Results ◽  
High Computational Cost

This paper proposes a framework that allows 3D freeform manipulation of a face in live video. Unlike existing approaches, the proposed framework provides natural 3D manipulation of a face without background distortion and interactive face editing by a user’s input, which leads to freeform manipulation without any limitation of range or shape. To achieve these features, a 3D morphable face model is fitted to a face region in a video frame and is deformed by the user’s input. The video frame is then mapped as a texture to the deformed model, and the model is rendered on the video frame. Because of the high computational cost, parallelization and acceleration schemes are also adopted for real-time performance. Performance evaluation and comparison results show that the proposed framework is promising for 3D face editing in live video.

scholarly journals HyP-DESPOT: A hybrid parallel algorithm for online planning under uncertainty

2020 ◽  
pp. 027836492093707
Author(s):  
Panpan Cai ◽  
Yuanfu Luo ◽  
David Hsu ◽  
Wee Sun Lee
Keyword(s):  
Real Time ◽  
Computational Cost ◽  
Search Tree ◽  
Planning Under Uncertainty ◽  
State Action ◽  
Online Planning ◽  
Planning Algorithm ◽  
One Step ◽  
Robotic Tasks ◽  
High Computational Cost

Robust planning under uncertainty is critical for robots in uncertain, dynamic environments, but incurs high computational cost. State-of-the-art online search algorithms, such as DESPOT, have vastly improved the computational efficiency of planning under uncertainty and made it a valuable tool for robotics in practice. This work takes one step further by leveraging both CPU and GPU parallelization in order to achieve real-time online planning performance for complex tasks with large state, action, and observation spaces. Specifically, Hybrid Parallel DESPOT (HyP-DESPOT) is a massively parallel online planning algorithm that integrates CPU and GPU parallelism in a multi-level scheme. It performs parallel DESPOT tree search by simultaneously traversing multiple independent paths using multi-core CPUs; it performs parallel Monte Carlo simulations at the leaf nodes of the search tree using GPUs. HyP-DESPOT provably converges in finite time under moderate conditions and guarantees near-optimality of the solution. Experimental results show that HyP-DESPOT speeds up online planning by up to a factor of several hundred in several challenging robotic tasks in simulation, compared with the original DESPOT algorithm. It also exhibits real-time performance on a robot vehicle navigating among many pedestrians.

scholarly journals Real-Time Haze Removal Using Normalised Pixel-Wise Dark-Channel Prior and Robust Atmospheric-Light Estimation

2020 ◽  
Vol 10 (3) ◽  
pp. 1165 ◽  
Author(s):  
Yutaro Iwamoto ◽  
Naoaki Hashimoto ◽  
Yen-Wei Chen
Keyword(s):  
Real Time ◽  
Computational Cost ◽  
Dark Channel Prior ◽  
Detailed Structure ◽  
Haze Removal ◽  
Acceptable Quality ◽  
Colour Channel ◽  
Dark Channel ◽  
High Computational Cost ◽  
Atmospheric Light

This study proposes real-time haze removal from a single image using normalised pixel-wise dark-channel prior (DCP). DCP assumes that at least one RGB colour channel within most local patches in a haze-free image has a low-intensity value. Since the spatial resolution of the transmission map depends on the patch size and it loses the detailed structure with large patch sizes, original work refines the transmission map using an image-matting technique. However, it requires high computational cost and is not adequate for real-time application. To solve these problems, we use normalised pixel-wise haze estimation without losing the detailed structure of the transmission map. This study also proposes robust atmospheric-light estimation using a coarse-to-fine search strategy and down-sampled haze estimation for acceleration. Experiments with actual and simulated haze images showed that the proposed method achieves real-time results of visually and quantitatively acceptable quality compared with other conventional methods of haze removal.

scholarly journals Proposta de Implementação em Hardware do Algoritmo de Otsu Aplicado ao Rastreamento em Tempo Real de Vermes

2021 ◽  
Author(s):  
Wysterlânya Kyury Pereira Barros ◽  
Marcelo Fernandes
Keyword(s):  
Rna Interference ◽  
High Resolution ◽  
Real Time ◽  
Computational Cost ◽  
Field Programmable ◽  
Otsu’S Method ◽  
Real Time Tracking ◽  
Global Thresholding ◽  
Thresholding Algorithm ◽  
High Computational Cost

This work proposes an implementation in Field Programmable GateArray (FPGA) of the Otsu’s method applied to real-time trackingof worms called Caenorhabditis elegans. Real-time tracking is necessaryto measure changes in the worm’s behavior in response totreatment with Ribonucleic Acid (RNA) interference. Otsu’s methodis a global thresholding algorithm used to define an optimal thresholdbetween two classes. However, this technique in real-time applicationsassociated with the processing of high-resolution videoshas a high computational cost because of the massive amount ofdata generated. Otsu’s algorithm needs to identify the worms ineach frame captured by a high-resolution camera in a real-timeanalysis of the worm’s behavior. Thus, this work proposes a highperformanceimplementation of Otsu’s algorithm in FPGA. Theresults show it was possible to achieve a speedup up to 5 timeshigher than similar works in the literature.

Interactive Sketch-Based Digital Prototyping by Using the Level-Set Method

2007 ◽  
Author(s):  
Weihan Zhang ◽  
Ming C. Leu
Keyword(s):  
Real Time ◽  
Level Set Method ◽  
Level Set ◽  
Computational Cost ◽  
Transformation Method ◽  
Volumetric Data ◽  
Shape Transformation ◽  
Time Performance ◽  
Virtual Prototypes ◽  
2D Sketches

This paper presents the development of a sketch-based interactive digital prototyping method by using the level-set method as the underlying technology. Various deformative operations (e.g., warping, smoothing) are developed by the level-set method to design 2D sketches. Fast and robust numerical techniques are utilized to support the real-time performance of level-set operations. B-spline function-based shape transformation method is developed to construct 3D volumetric data from multiple 2D sketches. Virtual prototypes are then visualized by reconstructing surface models from the volumetric data. Computational cost and memory requirement of the presented method are analyzed to evaluate its real-time performance. Example virtual prototypes are shown to demonstrate the capability of the developed method.

scholarly journals Real-Time Collision-Free Navigation of Multiple UAVs Based on Bounding Boxes

Electronics ◽  
2020 ◽  
Vol 9 (10) ◽  
pp. 1632
Author(s):  
Paloma Sánchez ◽  
Rafael Casado ◽  
Aurelio Bermúdez
Keyword(s):  
Real Time ◽  
Collision Avoidance ◽  
Computational Cost ◽  
Multiple Uavs ◽  
Navigation Algorithms ◽  
Excellent Candidate ◽  
Bounding Boxes ◽  
Velocity Obstacle ◽  
High Computational Cost ◽  
Multi Uav

Predictably, future urban airspaces will be crowded with autonomous unmanned aerial vehicles (UAVs) offering different services to the population. One of the main challenges in this new scenario is the design of collision-free navigation algorithms to avoid conflicts between flying UAVs. The most appropriate collision avoidance strategies for this scenario are non-centralized ones that are dynamically executed (in real time). Existing collision avoidance methods usually entail a high computational cost. In this work, we present Bounding Box Collision Avoidance (BBCA) algorithm, a simplified velocity obstacle-based technique that achieves a balance between efficiency and cost. The performance of the proposal is analyzed in detail in different airspace configurations. Simulation results show that the method is able to avoid all the conflicts in two UAV scenarios and most of them in multi-UAV ones. At the same time, we have found that the penalty of using the BBCA collision avoidance technique on the flying time and the distance covered by the UAVs involved in the conflict is reasonably acceptable. Therefore, we consider that BBCA may be an excellent candidate for the design of collision-free navigation algorithms for UAVs.

Pose recognition in sports scenes based on deep learning skeleton sequence model

2021 ◽  
pp. 1-10
Author(s):  
Chen Li-quan ◽  
Li You ◽  
Fengjun Shen ◽  
Zhaoqimeng Shan ◽  
Jiaxuan Chen
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Target Detection ◽  
Gesture Recognition ◽  
Video Frame ◽  
Skeleton Extraction ◽  
Recognition Method ◽  
Time Performance ◽  
Basic Network ◽  
Human Joints

Human skeleton extraction is a basic problem in the field of computer vision. With the rapid progress of science and technology, it has become a hot issue in the field of target detection such as pedestrian recognition, behavior monitoring, and pedestrian gesture recognition. In recent years, due to the development of deep neural networks, modeling of human joints in acquired images has made progress in skeleton extraction. However, most models have low modeling accuracy, poor real-time performance, and poor model availability. problem. Aiming at the above-mentioned human target detection problem, this paper uses the deep learning skeleton sequence model gesture recognition method in sports scenes to study, aiming to provide a gesture recognition method with strong noise resistance, good real-time performance and accurate model. This article uses motion video frame images to train the VGG16 network. Using the network to extract skeleton information can strengthen the posture feature expression, and use HOG for feature extraction, and use the Adam algorithm to optimize the network to extract more posture features, thereby improving the posture of the network Recognition accuracy. Then adjust the hyperparameters and network structure of the basic network according to the training results, and obtain the key poses in the sports scene through the final classifier.

An Experimental Evaluation of Debayering Algorithms on GPUs for Recording Panoramic Video in Real-Time

2015 ◽  
Vol 6 (3) ◽  
pp. 1-16 ◽  
Author(s):  
Ragnar Langseth ◽  
Vamsidhar Reddy Gaddam ◽  
Håkon Kvale Stensland ◽  
Carsten Griwodz ◽  
Pål Halvorsen ◽  
...  
Keyword(s):  
Real Time ◽  
Experimental Evaluation ◽  
Visual Quality ◽  
Video Frame ◽  
Post Processing ◽  
Live Video ◽  
Video Cameras ◽  
Video Recordings ◽  
Processing Step ◽  
Panoramic Video

Modern video cameras often only capture a single color per pixel in a single pass operation. This process is called ltering, where pixels are ltered through a color lter array, and the Bayer lter is perhaps the most common lter used today. This means that the missing color channels must be restored in the image or the video frame in a post-processing step, i.e., a process referred to as debayering. In a live video scenario, this operation must be performed eciently in order to output each video frame in real-time, while also yielding acceptable visual quality. Here, the authors evaluate debayering algorithms implemented on a GPU for real-time panoramic video recordings using multiple 2K-resolution cameras.

scholarly journals Fully Parallel Implementation of Otsu Automatic Image Thresholding Algorithm on FPGA

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4151
Author(s):  
Wysterlânya K. P. Barros ◽  
Leonardo A. Dias ◽  
Marcelo A. C. Fernandes
Keyword(s):  
Real Time ◽  
Parallel Implementation ◽  
Computational Cost ◽  
Image Thresholding ◽  
Otsu Method ◽  
Field Programmable ◽  
High Resolution Images ◽  
Area Occupation ◽  
Thresholding Algorithm ◽  
High Computational Cost

This work proposes a high-throughput implementation of the Otsu automatic image thresholding algorithm on Field Programmable Gate Array (FPGA), aiming to process high-resolution images in real-time. The Otsu method is a widely used global thresholding algorithm to define an optimal threshold between two classes. However, this technique has a high computational cost, making it difficult to use in real-time applications. Thus, this paper proposes a hardware design exploiting parallelization to optimize the system’s processing time. The implementation details and an analysis of the synthesis results concerning the hardware area occupation, throughput, and dynamic power consumption, are presented. Results have shown that the proposed hardware achieved a high speedup compared to similar works in the literature.

scholarly journals Improving Real-Time Hand Gesture Recognition with Semantic Segmentation

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 356
Author(s):  
Gibran Benitez-Garcia ◽  
Lidia Prudente-Tixteco ◽  
Luis Carlos Castro-Madrid ◽  
Rocio Toscano-Medina ◽  
Jesus Olivares-Mercado ◽  
...  
Keyword(s):  
Optical Flow ◽  
Real Time ◽  
Gesture Recognition ◽  
Computational Cost ◽  
Semantic Segmentation ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Temporal Shift ◽  
Time Performance ◽  
Wide Range

Hand gesture recognition (HGR) takes a central role in human–computer interaction, covering a wide range of applications in the automotive sector, consumer electronics, home automation, and others. In recent years, accurate and efficient deep learning models have been proposed for real-time applications. However, the most accurate approaches tend to employ multiple modalities derived from RGB input frames, such as optical flow. This practice limits real-time performance due to intense extra computational cost. In this paper, we avoid the optical flow computation by proposing a real-time hand gesture recognition method based on RGB frames combined with hand segmentation masks. We employ a light-weight semantic segmentation method (FASSD-Net) to boost the accuracy of two efficient HGR methods: Temporal Segment Networks (TSN) and Temporal Shift Modules (TSM). We demonstrate the efficiency of the proposal on our IPN Hand dataset, which includes thirteen different gestures focused on interaction with touchless screens. The experimental results show that our approach significantly overcomes the accuracy of the original TSN and TSM algorithms by keeping real-time performance.

Sign in / Sign up

Export Citation Format

Share Document