scholarly journals Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA

Sensors ◽  
2020 ◽  
Vol 20 (12) ◽  
pp. 3404 ◽  
Author(s):  
Ricardo Tapiador-Morales ◽  
Jean-Matthieu Maro ◽  
Angel Jimenez-Fernandez ◽  
Gabriel Jimenez-Moreno ◽  
Ryad Benosman ◽  
...  
Keyword(s):  
Gesture Recognition ◽  
Time History ◽  
High Temporal Resolution ◽  
Vision Sensors ◽  
Dynamic Vision ◽  
Mammalian Retina ◽  
Continuous Stream ◽  
Spatio Temporal ◽  
Event Based ◽  
Embedded Applications

Neuromorphic vision sensors detect changes in luminosity taking inspiration from mammalian retina and providing a stream of events with high temporal resolution, also known as Dynamic Vision Sensors (DVS). This continuous stream of events can be used to extract spatio-temporal patterns from a scene. A time-surface represents a spatio-temporal context for a given spatial radius around an incoming event from a sensor at a specific time history. Time-surfaces can be organized in a hierarchical way to extract features from input events using the Hierarchy Of Time-Surfaces algorithm, hereinafter HOTS. HOTS can be organized in consecutive layers to extract combination of features in a similar way as some deep-learning algorithms do. This work introduces a novel FPGA architecture for accelerating HOTS network. This architecture is mainly based on block-RAM memory and the non-restoring square root algorithm, requiring basic components and enabling it for low-power low-latency embedded applications. The presented architecture has been tested on a Zynq 7100 platform at 100 MHz. The results show that the latencies are in the range of 1 μ s to 6.7 μ s, requiring a maximum dynamic power consumption of 77 mW. This system was tested with a gesture recognition dataset, obtaining an accuracy loss for 16-bit precision of only 1.2% with respect to the original software HOTS.

A USB3.0 FPGA event-based filtering and tracking framework for dynamic vision sensors

2015 ◽  
Author(s):  
A. Linares-Barranco ◽  
F. Gomez-Rodriguez ◽  
V. Villanueva ◽  
L. Longinotti ◽  
T. Delbruck
Keyword(s):  
Vision Sensors ◽  
Dynamic Vision ◽  
Event Based

scholarly journals Spiking neural networks for computer vision

2018 ◽  
Vol 8 (4) ◽  
pp. 20180007 ◽  
Author(s):  
Michael Hopkins ◽  
Garibaldi Pineda-García ◽  
Petruţ A. Bogdan ◽  
Steve B. Furber
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Network Architecture ◽  
Visual Scene ◽  
Vision Systems ◽  
Biological Vision ◽  
Vision Sensors ◽  
Learning Capabilities ◽  
Spatio Temporal ◽  
Event Based

State-of-the-art computer vision systems use frame-based cameras that sample the visual scene as a series of high-resolution images. These are then processed using convolutional neural networks using neurons with continuous outputs. Biological vision systems use a quite different approach, where the eyes (cameras) sample the visual scene continuously, often with a non-uniform resolution, and generate neural spike events in response to changes in the scene. The resulting spatio-temporal patterns of events are then processed through networks of spiking neurons. Such event-based processing offers advantages in terms of focusing constrained resources on the most salient features of the perceived scene, and those advantages should also accrue to engineered vision systems based upon similar principles. Event-based vision sensors, and event-based processing exemplified by the SpiNNaker (Spiking Neural Network Architecture) machine, can be used to model the biological vision pathway at various levels of detail. Here we use this approach to explore structural synaptic plasticity as a possible mechanism whereby biological vision systems may learn the statistics of their inputs without supervision, pointing the way to engineered vision systems with similar online learning capabilities.

scholarly journals An Extended Modular Processing Pipeline for Event-Based Vision in Automatic Visual Inspection

Sensors ◽  
2021 ◽  
Vol 21 (18) ◽  
pp. 6143
Author(s):  
Moritz Beck ◽  
Georg Maier ◽  
Merle Flitter ◽  
Robin Gruna ◽  
Thomas Längle ◽  
...  
Keyword(s):  
Visual Inspection ◽  
Time Windows ◽  
Free Fall ◽  
Conveyor Belt ◽  
High Temporal Resolution ◽  
Event Stream ◽  
Practical Applications ◽  
Vision Sensors ◽  
Dynamic Vision ◽  
Automatic Visual Inspection

Dynamic Vision Sensors differ from conventional cameras in that only intensity changes of individual pixels are perceived and transmitted as an asynchronous stream instead of an entire frame. The technology promises, among other things, high temporal resolution and low latencies and data rates. While such sensors currently enjoy much scientific attention, there are only little publications on practical applications. One field of application that has hardly been considered so far, yet potentially fits well with the sensor principle due to its special properties, is automatic visual inspection. In this paper, we evaluate current state-of-the-art processing algorithms in this new application domain. We further propose an algorithmic approach for the identification of ideal time windows within an event stream for object classification. For the evaluation of our method, we acquire two novel datasets that contain typical visual inspection scenarios, i.e., the inspection of objects on a conveyor belt and during free fall. The success of our algorithmic extension for data processing is demonstrated on the basis of these new datasets by showing that classification accuracy of current algorithms is highly increased. By making our new datasets publicly available, we intend to stimulate further research on application of Dynamic Vision Sensors in machine vision applications.

scholarly journals Low Latency Event-Based Filtering and Feature Extraction for Dynamic Vision Sensors in Real-Time FPGA Applications

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 134926-134942 ◽  
Author(s):  
Alejandro Linares-Barranco ◽  
Fernando Perez-Pena ◽  
Diederik Paul Moeys ◽  
Francisco Gomez-Rodriguez ◽  
Gabriel Jimenez-Moreno ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Real Time ◽  
Low Latency ◽  
Vision Sensors ◽  
Dynamic Vision ◽  
Event Based
Sign in / Sign up

Export Citation Format

Share Document