Siamese Network Object Tracking Algorithm Combining Attention Mechanism and Correlation Filter Theory

Aiming to solve the problem of tracking drift during movement, which was caused by the lack of discriminability of the feature information and the failure of a fixed template to adapt to the change of object appearance, the paper proposes an object tracking algorithm combining attention mechanism and correlation filter theory based on the framework of full convolutional Siamese neural networks. Firstly, the apparent information is processed by using the attention mechanism thought, where the object and search area features are optimized according to the spatial attention and channel attention module. At the same time, the cross-attention module is introduced to process the template branch and search area branch, respectively, which makes full use of the diversified context information of the search area. Then, the background perception correlation filter model with scale adaptation and learning rate adjustment is adopted into the model construction, using as a layer in the network model to realize the object template update. Finally, the optimal object location is determined according to the confidence map with similarity calculation. Experimental results show that the designed method in the paper can promote the object tracking performance under various challenging environments effectively; the success rate increases by 16.2%, and the accuracy rate increases by 16%.

Robust Data Association Using Fusion of Data-Driven and Engineered Features for Real-Time Pedestrian Tracking in Thermal Images

Object tracking is an essential problem in computer vision that has been extensively researched for decades. Tracking objects in thermal images is particularly difficult because of the lack of color information, low image resolution, or high similarity between objects of the same class. One of the main challenges in multi-object tracking, also referred to as the data association problem, is finding the correct correspondences between measurements and tracks and adapting the object appearance changes over time. We addressed this challenge of data association for thermal images by proposing three contributions. The first contribution consisted of the creation of a data-driven appearance score using five Siamese Networks, which operate on the image detection and on parts of it. Secondly, we engineered an original edge-based descriptor that improves the data association process. Lastly, we proposed a dataset consisting of pedestrian instances that were recorded in different scenarios and are used for training the Siamese Networks. The data-driven part of the data association score offers robustness, while feature engineering offers adaptability to unknown scenarios and their combination leads to a more powerful tracking solution. Our approach had a running time of 25 ms and achieved an average precision of 86.2% on publicly available benchmarks, containing real-world scenarios, as shown in the evaluation section.

Robust image-based risk predictions from the deep learning of lung tumors in motion

Deep learning (DL) models that use medical images to predict clinical outcomes are poised for clinical translation. For tumors that reside in organs that move, however, the impact of motion (i.e. degenerated object appearance or blur) on DL model accuracy remains unclear. Here we examine the impact of tumor motion on an image-based DL framework that predicts local failure risk for patients with lung cancer receiving stereotactic body radiotherapy. We show that an image-based DL risk score derived from a series of four-dimensional CT images varies in a deterministic, sinusoidal trajectory in phase with the respiratory cycle. Critically, the mean of the scores derived from time series of images and the score obtained from free breathing scans (average tumor position) were highly associated (Pearson r = 0.99). These results indicate that deep learning models of tumors in motion can be robust to fluctuations in object appearance due to movement.

Psychophysical evidence and perceptual observations show that object recognition is not hierarchical but is a parallel, simultaneous, egalitarian, non-computational system.

Object recognition models have at their core similar essential characteristics: feature extraction and hierarchical convergence leading to a code that is unique to each object and immune to variations in the object appearance. To compare computational, biologically-feasible models to human performance, subjects viewed objects displayed at a wide range of orientations and sizes, and were able to recognize them almost perfectly. These empirical results, together with consideration of thought experiments and analysis of everyday perceptual performance, lead to a conclusion that biologically-plausible object perception models do not even come close to matching our perceptual abilities. We can categorize many thousands of objects, discriminate between enormous numbers of different exemplars within each category, and recognize an object as unique although it may appear in countless variations—most of which have never been seen. This seemingly technical, quantitative failure stems from a fundamental property of our perception: the ability to perceive spatial information instantaneously and in parallel, retain details including their relative properties, and yet be able to integrate details into a meaningful percept such as an object. I present an alternative view of object perception whereby objects are represented by responses in primary visual cortex (V1) which is the only cortical area responding to small spatial elements. The rest of the visual cortex is dedicated to scene understanding and interpretation such as constructing 3D percepts from 2D inputs, coding motion, categorization and memories. Since our perception abilities cannot be explained by convergence to 'object cells' or by interactions implemented by axonal transmissions, a parallel-to-parallel field-like process is suggested. In this view, spatial information is not modified by multiple neural interactions but is retained by affecting changes in a 'neural field' which preserves the identity of individual elements while enabling a new holistic percept when these elements change.

OBJECT RE-IDENTIFICATION USING MULTIMODAL AERIAL IMAGERY AND CONDITIONAL ADVERSARIAL NETWORKS

Object Re-Identification (ReID) is the task of matching a given object in the new environment with its image captured in a different environment. The input for a ReID method includes two sets of images. The probe set includes one or more images of the object that must be identified in the new environment. The gallery set includes images that may contain the object from the probe image. The ReID task’s complexity arises from the differences in the object appearance in the probe and gallery sets. Such difference may originate from changes in illumination or viewpoint locations for multiple cameras that capture images in the probe and gallery sets. This paper focuses on developing a deep learning ThermalReID framework for cross-modality object ReID in thermal images. Our framework aims to provide continuous object detection and re-identification while monitoring a region from a UAV. Given an input probe image captured in the visible range, our ThermalReID framework detects objects in a thermal image and performs the ReID. We evaluate our ThermalReID framework and modern baselines using various metrics. We use the IoU and mAP metrics for the object detection task. We use the cumulative matching characteristic (CMC) curves and normalized area-under-curve (nAUC) for the ReID task. The evaluation demonstrated encouraging results and proved that our ThermalReID framework outperforms existing baselines in the ReID accuracy. Furthermore, we demonstrated that the fusion of the semantic data with the input thermal gallery image increases the object detection and localization scores. We developed the ThermalReID framework for cross-modality object re-identification. We evaluated our framework and two modern baselines on the task of object ReID for four object classes. Our framework successfully performs object ReID in the thermal gallery image from the color probe image. The evaluation using real and synthetic data demonstrated that our ThermalReID framework increases the ReID accuracy compared to modern ReID baselines.

A Robust Quadruplet and Faster Region-Based CNN for UAV Video-Based Multiple Object Tracking in Crowded Environment

Multiple object tracking (MOT) from unmanned aerial vehicle (UAV) videos has faced several challenges such as motion capture and appearance, clustering, object variation, high altitudes, and abrupt motion. Consequently, the volume of objects captured by the UAV is usually quite small, and the target object appearance information is not always reliable. To solve these issues, a new technique is presented to track objects based on a deep learning technique that attains state-of-the-art performance on standard datasets, such as Stanford Drone and Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking (UAVDT) datasets. The proposed faster RCNN (region-based convolutional neural network) framework was enhanced by integrating a series of activities, including the proper calibration of key parameters, multi-scale training, hard negative mining, and feature collection to improve the region-based CNN baseline. Furthermore, a deep quadruplet network (DQN) was applied to track the movement of the captured objects from the crowded environment, and it was modelled to utilize new quadruplet loss function in order to study the feature space. A deep 6 Rectified linear units (ReLU) convolution was used in the faster RCNN to mine spatial–spectral features. The experimental results on the standard datasets demonstrated a high performance accuracy. Thus, the proposed method can be used to detect multiple objects and track their trajectories with a high accuracy.

Learning Local–Global Multiple Correlation Filters for Robust Visual Tracking with Kalman Filter Redetection

Visual object tracking is a significant technology for camera-based sensor networks applications. Multilayer convolutional features comprehensively used in correlation filter (CF)-based tracking algorithms have achieved excellent performance. However, there are tracking failures in some challenging situations because ordinary features are not able to well represent the object appearance variations and the correlation filters are updated irrationally. In this paper, we propose a local–global multiple correlation filters (LGCF) tracking algorithm for edge computing systems capturing moving targets, such as vehicles and pedestrians. First, we construct a global correlation filter model with deep convolutional features, and choose horizontal or vertical division according to the aspect ratio to build two local filters with hand-crafted features. Then, we propose a local–global collaborative strategy to exchange information between local and global correlation filters. This strategy can avoid the wrong learning of the object appearance model. Finally, we propose a time-space peak to sidelobe ratio (TSPSR) to evaluate the stability of the current CF. When the estimated results of the current CF are not reliable, the Kalman filter redetection (KFR) model would be enabled to recapture the object. The experimental results show that our presented algorithm achieves better performances on OTB-2013 and OTB-2015 compared with the other latest 12 tracking algorithms. Moreover, our algorithm handles various challenges in object tracking well.

Deep Convolutional Neural Network for Object Classification

Object classification problem is essential in many applications nowadays. Human can easily classify objects in unconstrained environments easily. Classical classification techniques were far away from human performance. Thus, researchers try to mimic the human visual system till they reached the deep neural networks. This chapter gives a review and analysis in the field of the deep convolutional neural network usage in object classification under constrained and unconstrained environment. The chapter gives a brief review on the classical techniques of object classification and the development of bio-inspired computational models from neuroscience till the creation of deep neural networks. A review is given on the constrained environment issues: the hardware computing resources and memory, the object appearance and background, and the training and processing time. Datasets that are used to test the performance are analyzed according to the images environmental conditions, besides the dataset biasing is discussed.