scholarly journals Radar Voxel Fusion for 3D Object Detection

2021 ◽  
Vol 11 (12) ◽  
pp. 5598
Author(s):  
Felix Nobis ◽  
Ehsan Shafiei ◽  
Phillip Karle ◽  
Johannes Betz ◽  
Markus Lienkamp
Keyword(s):  
Data Fusion ◽  
Object Detection ◽  
Sensor Fusion ◽  
Weather Conditions ◽  
Radar Data ◽  
Sensor Data ◽  
Data Set ◽  
3D Object ◽  
Individual Object ◽  
3D Object Detection

Automotive traffic scenes are complex due to the variety of possible scenarios, objects, and weather conditions that need to be handled. In contrast to more constrained environments, such as automated underground trains, automotive perception systems cannot be tailored to a narrow field of specific tasks but must handle an ever-changing environment with unforeseen events. As currently no single sensor is able to reliably perceive all relevant activity in the surroundings, sensor data fusion is applied to perceive as much information as possible. Data fusion of different sensors and sensor modalities on a low abstraction level enables the compensation of sensor weaknesses and misdetections among the sensors before the information-rich sensor data are compressed and thereby information is lost after a sensor-individual object detection. This paper develops a low-level sensor fusion network for 3D object detection, which fuses lidar, camera, and radar data. The fusion network is trained and evaluated on the nuScenes data set. On the test set, fusion of radar data increases the resulting AP (Average Precision) detection score by about 5.1% in comparison to the baseline lidar network. The radar sensor fusion proves especially beneficial in inclement conditions such as rain and night scenes. Fusing additional camera data contributes positively only in conjunction with the radar fusion, which shows that interdependencies of the sensors are important for the detection result. Additionally, the paper proposes a novel loss to handle the discontinuity of a simple yaw representation for object detection. Our updated loss increases the detection and orientation estimation performance for all sensor input configurations. The code for this research has been made available on GitHub.

scholarly journals One-Stage Multi-Sensor Data Fusion Convolutional Neural Network for 3D Object Detection

Sensors ◽  
2019 ◽  
Vol 19 (6) ◽  
pp. 1434 ◽  
Author(s):  
Minle Li ◽  
Yihua Hu ◽  
Nanxiang Zhao ◽  
Qishu Qian
Keyword(s):  
Neural Network ◽  
Feature Extraction ◽  
Data Fusion ◽  
Object Detection ◽  
Sensor Data ◽  
3D Object ◽  
Sensor Data Fusion ◽  
One Stage ◽  
Multi Sensor Data Fusion ◽  
3D Object Detection

Three-dimensional (3D) object detection has important applications in robotics, automatic loading, automatic driving and other scenarios. With the improvement of devices, people can collect multi-sensor/multimodal data from a variety of sensors such as Lidar and cameras. In order to make full use of various information advantages and improve the performance of object detection, we proposed a Complex-Retina network, a convolution neural network for 3D object detection based on multi-sensor data fusion. Firstly, a unified architecture with two feature extraction networks was designed, and the feature extraction of point clouds and images from different sensors realized synchronously. Then, we set a series of 3D anchors and projected them to the feature maps, which were cropped into 2D anchors with the same size and fused together. Finally, the object classification and 3D bounding box regression were carried out on the multipath of fully connected layers. The proposed network is a one-stage convolution neural network, which achieves the balance between the accuracy and speed of object detection. The experiments on KITTI datasets show that the proposed network is superior to the contrast algorithms in average precision (AP) and time consumption, which shows the effectiveness of the proposed network.

scholarly journals Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation

2021 ◽  
Vol 11 (6) ◽  
pp. 2599
Author(s):  
Felix Nobis ◽  
Felix Fent ◽  
Johannes Betz ◽  
Markus Lienkamp
Keyword(s):  
Object Detection ◽  
Point Cloud ◽  
State Of The Art ◽  
Radar Data ◽  
Quality Data ◽  
Data Set ◽  
3D Object ◽  
The Public ◽  
3D Object Detection

State-of-the-art 3D object detection for autonomous driving is achieved by processing lidar sensor data with deep-learning methods. However, the detection quality of the state of the art is still far from enabling safe driving in all conditions. Additional sensor modalities need to be used to increase the confidence and robustness of the overall detection result. Researchers have recently explored radar data as an additional input source for universal 3D object detection. This paper proposes artificial neural network architectures to segment sparse radar point cloud data. Segmentation is an intermediate step towards radar object detection as a complementary concept to lidar object detection. Conceptually, we adapt Kernel Point Convolution (KPConv) layers for radar data. Additionally, we introduce a long short-term memory (LSTM) variant based on KPConv layers to make use of the information content in the time dimension of radar data. This is motivated by classical radar processing, where tracking of features over time is imperative to generate confident object proposals. We benchmark several variants of the network on the public nuScenes data set against a state-of-the-art pointnet-based approach. The performance of the networks is limited by the quality of the publicly available data. The radar data and radar-label quality is of great importance to the training and evaluation of machine learning models. Therefore, the advantages and disadvantages of the available data set, regarding its radar data, are discussed in detail. The need for a radar-focused data set for object detection is expressed. We assume that higher segmentation scores should be achievable with better-quality data for all models compared, and differences between the models should manifest more clearly. To facilitate research with additional radar data, the modular code for this research will be made available to the public.

Deep Sensor Fusion Based on Frustum Point Single Shot Multibox Detector for 3D Object Detection

2021 ◽  
Author(s):  
Yu Wang ◽  
Ye Zhang ◽  
Shaohua Zhai ◽  
Hao Chen ◽  
Shaoqi Shi ◽  
...  
Keyword(s):  
Object Detection ◽  
Sensor Fusion ◽  
Single Shot ◽  
3D Object ◽  
3D Object Detection

scholarly journals 3D-FFS: Faster 3D object detection with Focused Frustum Search in sensor fusion based networks

2021 ◽  
Author(s):  
Aniruddha Ganguly ◽  
Tasin Ishmam ◽  
Khandker Aftarul Islam ◽  
Md Zahidur Rahman ◽  
Md. Shamsuzzoha Bayzid
Keyword(s):  
Object Detection ◽  
Sensor Fusion ◽  
3D Object ◽  
3D Object Detection
Sign in / Sign up

Export Citation Format

Share Document