Spatio-temporal Attention Mechanism for More Complex Analysis to Track Multiple Objects

2005 ◽  
pp. 447-456 ◽  
Author(s):  
Heungkyu Lee ◽  
Hanseok Ko
Keyword(s):  
Complex Analysis ◽  
Attention Mechanism ◽  
Temporal Attention ◽  
Multiple Objects ◽  
Spatio Temporal

PREDICTION BASED OCCLUDED MULTITARGET TRACKING USING SPATIO-TEMPORAL ATTENTION

2006 ◽  
Vol 20 (06) ◽  
pp. 925-938 ◽  
Author(s):  
HEUNGKYU LEE ◽  
JUNE KIM ◽  
HANSEOK KO
Keyword(s):  
Complex Analysis ◽  
Probability Model ◽  
Multitarget Tracking ◽  
Activity Detection ◽  
Temporal Attention ◽  
Missing Information ◽  
Multiple Objects ◽  
Partial Probability ◽  
Spatio Temporal ◽  
Object Association

This paper proposes the prediction based occluded multitarget tracking method using spatio-temporal attention mechanism. To cope with occlusion between targets, the proposed method provides an efficient method for more complex analysis by combining object association with partial probability model in spatially attentive window and occlusion activity detection in predicted temporal location. While multiple objects are moving or occluding between them in areas of visual field, a simultaneous tracking of multiple objects tends to fail. This is due to the fact that incompletely estimated feature vectors such as location, color, velocity, and acceleration of a target can provide only ambiguous and missing information. Thus, the spatially and temporally considered mechanism is proposed to track each target before, during, and after occlusion. Robustness of the proposed method is demonstrated with representative simulations.

Multi-Head Spatio-Temporal Attention Mechanism for Urban Anomaly Event Prediction

2021 ◽  
Vol 5 (3) ◽  
pp. 1-21
Author(s):  
Huiqun Huang ◽  
Xi Yang ◽  
Suining He
Keyword(s):  
New York ◽  
Experimental Studies ◽  
Attention Mechanism ◽  
City Management ◽  
Temporal Attention ◽  
Spatial Correlations ◽  
Time Step ◽  
Event Prediction ◽  
Spatio Temporal ◽  
Prediction Approach

Timely forecasting the urban anomaly events in advance is of great importance to the city management and planning. However, anomaly event prediction is highly challenging due to the sparseness of data, geographic heterogeneity (e.g., complex spatial correlation, skewed spatial distribution of anomaly events and crowd flows), and the dynamic temporal dependencies. In this study, we propose M-STAP, a novel Multi-head Spatio-Temporal Attention Prediction approach to address the problem of multi-region urban anomaly event prediction. Specifically, M-STAP considers the problem from three main aspects: (1) extracting the spatial characteristics of the anomaly events in different regions, and the spatial correlations between anomaly events and crowd flows; (2) modeling the impacts of crowd flow dynamic of the most relevant regions in each time step on the anomaly events; and (3) employing attention mechanism to analyze the varying impacts of the historical anomaly events on the predicted data. We have conducted extensive experimental studies on the crowd flows and anomaly events data of New York City, Melbourne and Chicago. Our proposed model shows higher accuracy (41.91% improvement on average) in predicting multi-region anomaly events compared with the state-of-the-arts.

scholarly journals Spatial–Temporal Attention Two-Stream Convolution Neural Network for Smoke Region Detection

Fire ◽  
2021 ◽  
Vol 4 (4) ◽  
pp. 66
Author(s):  
Zhipeng Ding ◽  
Yaqin Zhao ◽  
Ao Li ◽  
Zhaoxiang Zheng
Keyword(s):  
Neural Network ◽  
Image Classification ◽  
Fire Behavior ◽  
Flow Characteristics ◽  
Attention Mechanism ◽  
Detection Accuracy ◽  
Temporal Attention ◽  
Multiple Perspectives ◽  
Region Detection ◽  
Spatio Temporal

Smoke detection is of great significance for fire location and fire behavior analysis in a fire video surveillance system. Smoke image classification methods based on a deep convolution network have achieved high accuracy. However, the combustion of different types of fuel can produce smoke with different colors, such as black smoke, grey smoke, and white smoke. Additionally, the diffusion characteristic of smoke can lead to transparent smoke regions accompanied by colors and textures of background objects. Therefore, compared with smoke image classification, smoke region detection is a challenging task. This paper proposes a two-stream convolutional neural network based on spatio-temporal attention mechanism for smoke region segmentation (STCNNsmoke). The spatial stream extracts spatial features of foreground objects using the semi-supervised ranking model. The temporal stream uses optical flow characteristics to represent the dynamic characteristics of smoke such as diffusion and flutter features. Specifically, the spatio-temporal attention mechanism is presented to fuse the spatial and temporal characteristics of smoke and pay more attention to the moving regions with smoke colors and textures by predicting attention weights of channels. Furthermore, the spatio-temporal attention model improves the channel response of smoke-moving regions for the segmentation of complete smoke regions. The proposed method is evaluated and analyzed from multiple perspectives such as region detection accuracy and anti-interference. The experimental results showed that the proposed method significantly improved the ability of segmenting thin smoke and small smoke.

scholarly journals An Integrated Perspective on Spatio-Temporal Attention and Infant Language Acquisition

2021 ◽  
Vol 18 (4) ◽  
pp. 1592
Author(s):  
Sofia Russo ◽  
Giulia Calignano ◽  
Marco Dispaldro ◽  
Eloisa Valenza
Keyword(s):  
Visual Field ◽  
Speech Signal ◽  
Discrimination Task ◽  
Language Skills ◽  
Visual Stimuli ◽  
Temporal Attention ◽  
Early Language ◽  
Attentional Processes ◽  
Spatio Temporal ◽  
Infant Language

Efficiency in the early ability to switch attention toward competing visual stimuli (spatial attention) may be linked to future ability to detect rapid acoustic changes in linguistic stimuli (temporal attention). To test this hypothesis, we compared individual performances in the same cohort of Italian-learning infants in two separate tasks: (i) an overlap task, measuring disengagement efficiency for visual stimuli at 4 months (Experiment 1), and (ii) an auditory discrimination task for trochaic syllabic sequences at 7 months (Experiment 2). Our results indicate that an infant’s efficiency in processing competing information in the visual field (i.e., visuospatial attention; Exp. 1) correlates with the subsequent ability to orient temporal attention toward relevant acoustic changes in the speech signal (i.e., temporal attention; Exp. 2). These results point out the involvement of domain-general attentional processes (not specific to language or the sensorial domain) playing a pivotal role in the development of early language skills in infancy.

Sign in / Sign up

Export Citation Format

Share Document