3D object detection based on sparse convolution neural network and feature fusion for autonomous driving in smart cities

2020 ◽  
Vol 54 ◽  
pp. 102002 ◽  
Author(s):  
Lei Wang ◽  
Xiaoyun Fan ◽  
Jiahao Chen ◽  
Jun Cheng ◽  
Jun Tan ◽  
...  
Keyword(s):  
Neural Network ◽  
Object Detection ◽  
Feature Fusion ◽  
Smart Cities ◽  
Autonomous Driving ◽  
Convolution Neural Network ◽  
3D Object ◽  
3D Object Detection

scholarly journals RCBi-CenterNet: An Absolute Pose Policy for 3D Object Detection in Autonomous Driving

2021 ◽  
Vol 11 (12) ◽  
pp. 5621
Author(s):  
Kang An ◽  
Yixin Chen ◽  
Suhong Wang ◽  
Zhifeng Xiao
Keyword(s):  
Object Detection ◽  
Data Augmentation ◽  
Feature Fusion ◽  
Autonomous Driving ◽  
Future Research ◽  
3D Object ◽  
Center Point ◽  
Perception System ◽  
The Absolute ◽  
3D Object Detection

3D Object detection is a critical mission of the perception system of a self-driving vehicle. Existing bounding box-based methods are hard to train due to the need to remove duplicated detections in the post-processing stage. In this paper, we propose a center point-based deep neural network (DNN) architecture named RCBi-CenterNet that predicts the absolute pose for each detected object in the 3D world space. RCBi-CenterNet is composed of a recursive composite network with a dual-backbone feature extractor and a bi-directional feature pyramid network (BiFPN) for cross-scale feature fusion. In the detection head, we predict a confidence heatmap that is used to determine the position of detected objects. The other pose information, including depth and orientation, is regressed. We conducted extensive experiments on the Peking University/Baidu-Autonomous Driving dataset, which contains more than 60,000 labeled 3D vehicle instances from 5277 real-world images, and each vehicle object is annotated with the absolute pose described by the six degrees of freedom (6DOF). We validated the design choices of various data augmentation methods and the backbone options. Through an ablation study and an overall comparison with the state-of-the-art (SOTA), namely CenterNet, we showed that the proposed RCBi-CenterNet presents performance gains of 2.16%, 2.76%, and 5.24% in Top 1, Top 3, and Top 10 mean average precision (mAP). The model and the result could serve as a credible benchmark for future research in center point-based object detection.

scholarly journals A Two-Stage Data Association Approach for 3D Multi-Object Tracking

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 2894
Author(s):  
Minh-Quan Dao ◽  
Vincent Frémont
Keyword(s):  
Object Detection ◽  
Object Tracking ◽  
Moving Objects ◽  
Data Association ◽  
Autonomous Driving ◽  
Tracking Accuracy ◽  
Two Stage ◽  
Bipartite Matching ◽  
3D Object ◽  
3D Object Detection

Multi-Object Tracking (MOT) is an integral part of any autonomous driving pipelines because it produces trajectories of other moving objects in the scene and predicts their future motion. Thanks to the recent advances in 3D object detection enabled by deep learning, track-by-detection has become the dominant paradigm in 3D MOT. In this paradigm, a MOT system is essentially made of an object detector and a data association algorithm which establishes track-to-detection correspondence. While 3D object detection has been actively researched, association algorithms for 3D MOT has settled at bipartite matching formulated as a Linear Assignment Problem (LAP) and solved by the Hungarian algorithm. In this paper, we adapt a two-stage data association method which was successfully applied to image-based tracking to the 3D setting, thus providing an alternative for data association for 3D MOT. Our method outperforms the baseline using one-stage bipartite matching for data association by achieving 0.587 Average Multi-Object Tracking Accuracy (AMOTA) in NuScenes validation set and 0.365 AMOTA (at level 2) in Waymo test set.

scholarly journals Strong-Weak Feature Alignment for 3D Object Detection

Electronics ◽  
2021 ◽  
Vol 10 (10) ◽  
pp. 1205
Author(s):  
Zhiyu Wang ◽  
Li Wang ◽  
Bin Dai
Keyword(s):  
Object Detection ◽  
Point Clouds ◽  
Autonomous Driving ◽  
Feature Representation ◽  
Alignment Algorithm ◽  
3D Object ◽  
3D Point Clouds ◽  
Object Feature ◽  
3D Object Detection ◽  
Feature Alignment

Object detection in 3D point clouds is still a challenging task in autonomous driving. Due to the inherent occlusion and density changes of the point cloud, the data distribution of the same object will change dramatically. Especially, the incomplete data with sparsity or occlusion can not represent the complete characteristics of the object. In this paper, we proposed a novel strong–weak feature alignment algorithm between complete and incomplete objects for 3D object detection, which explores the correlations within the data. It is an end-to-end adaptive network that does not require additional data and can be easily applied to other object detection networks. Through a complete object feature extractor, we achieve a robust feature representation of the object. It serves as a guarding feature to help the incomplete object feature generator to generate effective features. The strong–weak feature alignment algorithm reduces the gap between different states of the same object and enhances the ability to represent the incomplete object. The proposed adaptation framework is validated on the KITTI object benchmark and gets about 6% improvement in detection average precision on 3D moderate difficulty compared to the basic model. The results show that our adaptation method improves the detection performance of incomplete 3D objects.

Real-Time 3D object detection using improved convolutional neural network based on image-driven point cloud

2021 ◽  
Vol 14 ◽  
Author(s):  
Zhiyong Gao ◽  
Jianhong Xiang
Keyword(s):  
Neural Network ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Real Time ◽  
Point Cloud ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Object ◽  
3D Object Detection ◽  
Instance Segmentation

Background: While detecting the object directly from the 3D point cloud, the natural 3D patterns and invariance of 3D data are often obscure. Objective: In this work, we aimed at studying the 3D object detection from discrete, disordered and sparse 3D point clouds. Methods: The CNN is composed of the frustum sequence module, 3D instance segmentation module S-NET, 3D point cloud transformation module T-NET, and 3D boundary box estimation module E-NET. The search space of the object is determined by the frustum sequence module. The instance segmentation of the point cloud is performed by the 3D instance segmentation module. The 3D coordinates of the object are confirmed by the transformation module and the 3D bounding box estimation module. Results: Evaluated on KITTI benchmark dataset, our method outperforms the state of the art by remarkable margins while having real-time capability. Conclusion: We achieve real-time 3D object detection by proposing an improved convolutional neural network (CNN) based on image-driven point clouds.

Feature Fusion based Re-voting for 3D Object Detection

2021 ◽  
Author(s):  
Hang Yu ◽  
Jun Wei ◽  
Jinhe Su ◽  
Niansheng Liu
Keyword(s):  
Object Detection ◽  
Feature Fusion ◽  
3D Object ◽  
3D Object Detection

scholarly journals 3D Object Detection Algorithm for Panoramic Images With Multi-Scale Convolutional Neural Network

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 171461-171470
Author(s):  
Dianwei Wang ◽  
Yanhui He ◽  
Ying Liu ◽  
Daxiang Li ◽  
Shiqian Wu ◽  
...  
Keyword(s):  
Neural Network ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Detection Algorithm ◽  
3D Object ◽  
Multi Scale ◽  
Panoramic Images ◽  
3D Object Detection

Monocular 3D Object Detection for Autonomous Driving

2016 ◽  
Author(s):  
Xiaozhi Chen ◽  
Kaustav Kundu ◽  
Ziyu Zhang ◽  
Huimin Ma ◽  
Sanja Fidler ◽  
...  
Keyword(s):  
Object Detection ◽  
Autonomous Driving ◽  
3D Object ◽  
3D Object Detection

scholarly journals One-Stage Multi-Sensor Data Fusion Convolutional Neural Network for 3D Object Detection

Sensors ◽  
2019 ◽  
Vol 19 (6) ◽  
pp. 1434 ◽  
Author(s):  
Minle Li ◽  
Yihua Hu ◽  
Nanxiang Zhao ◽  
Qishu Qian
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Data Fusion ◽  
Object Detection ◽  
Sensor Data ◽  
3D Object ◽  
Sensor Data Fusion ◽  
One Stage ◽  
Multi Sensor Data Fusion ◽  
3D Object Detection

Three-dimensional (3D) object detection has important applications in robotics, automatic loading, automatic driving and other scenarios. With the improvement of devices, people can collect multi-sensor/multimodal data from a variety of sensors such as Lidar and cameras. In order to make full use of various information advantages and improve the performance of object detection, we proposed a Complex-Retina network, a convolution neural network for 3D object detection based on multi-sensor data fusion. Firstly, a unified architecture with two feature extraction networks was designed, and the feature extraction of point clouds and images from different sensors realized synchronously. Then, we set a series of 3D anchors and projected them to the feature maps, which were cropped into 2D anchors with the same size and fused together. Finally, the object classification and 3D bounding box regression were carried out on the multipath of fully connected layers. The proposed network is a one-stage convolution neural network, which achieves the balance between the accuracy and speed of object detection. The experiments on KITTI datasets show that the proposed network is superior to the contrast algorithms in average precision (AP) and time consumption, which shows the effectiveness of the proposed network.

scholarly journals Convolutional Neural Network Using for Multi-Sensor 3D Object Detection

2021 ◽  
Vol 1979 (1) ◽  
pp. 012020
Author(s):  
Gadug Sudhansu ◽  
A N Mohamed Zabeeulla ◽  
M N Nachappa
Keyword(s):  
Neural Network ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
3D Object ◽  
3D Object Detection

R-CNN Based 3D Object Detection for Autonomous Driving

CICTP 2020 ◽  
2020 ◽  
Author(s):  
Hongyu Hu ◽  
Tongtong Zhao ◽  
Qi Wang ◽  
Fei Gao ◽  
Lei He
Keyword(s):  
Object Detection ◽  
Autonomous Driving ◽  
3D Object ◽  
3D Object Detection
Sign in / Sign up

Export Citation Format

Share Document