scholarly journals Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation

2019 ◽  
Author(s):  
Gregory P. Meyer ◽  
Jake Charland ◽  
Darshan Hegde ◽  
Ankit Laddha ◽  
Carlos Vallespi-Gonzalez
Keyword(s):  
Object Detection ◽  
Sensor Fusion ◽  
Semantic Segmentation ◽  
3D Object ◽  
3D Object Detection

Deep Sensor Fusion Based on Frustum Point Single Shot Multibox Detector for 3D Object Detection

2021 ◽  
Author(s):  
Yu Wang ◽  
Ye Zhang ◽  
Shaohua Zhai ◽  
Hao Chen ◽  
Shaoqi Shi ◽  
...  
Keyword(s):  
Object Detection ◽  
Sensor Fusion ◽  
Single Shot ◽  
3D Object ◽  
3D Object Detection

scholarly journals Multi-View Fusion-Based 3D Object Detection for Robot Indoor Scene Perception

Sensors ◽  
2019 ◽  
Vol 19 (19) ◽  
pp. 4092 ◽  
Author(s):  
Li Wang ◽  
Ruifeng Li ◽  
Jingwen Sun ◽  
Xingxing Liu ◽  
Lijun Zhao ◽  
...  
Keyword(s):  
Object Detection ◽  
Scene Perception ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Service Robot ◽  
Multiple Views ◽  
Object Point ◽  
3D Object ◽  
Bounding Box ◽  
3D Object Detection

To autonomously move and operate objects in cluttered indoor environments, a service robot requires the ability of 3D scene perception. Though 3D object detection can provide an object-level environmental description to fill this gap, a robot always encounters incomplete object observation, recurring detections of the same object, error in detection, or intersection between objects when conducting detection continuously in a cluttered room. To solve these problems, we propose a two-stage 3D object detection algorithm which is to fuse multiple views of 3D object point clouds in the first stage and to eliminate unreasonable and intersection detections in the second stage. For each view, the robot performs a 2D object semantic segmentation and obtains 3D object point clouds. Then, an unsupervised segmentation method called Locally Convex Connected Patches (LCCP) is utilized to segment the object accurately from the background. Subsequently, the Manhattan Frame estimation is implemented to calculate the main orientation of the object and subsequently, the 3D object bounding box can be obtained. To deal with the detected objects in multiple views, we construct an object database and propose an object fusion criterion to maintain it automatically. Thus, the same object observed in multi-view is fused together and a more accurate bounding box can be calculated. Finally, we propose an object filtering approach based on prior knowledge to remove incorrect and intersecting objects in the object dataset. Experiments are carried out on both SceneNN dataset and a real indoor environment to verify the stability and accuracy of 3D semantic segmentation and bounding box detection of the object with multi-view fusion.

scholarly journals Radar Voxel Fusion for 3D Object Detection

2021 ◽  
Vol 11 (12) ◽  
pp. 5598
Author(s):  
Felix Nobis ◽  
Ehsan Shafiei ◽  
Phillip Karle ◽  
Johannes Betz ◽  
Markus Lienkamp
Keyword(s):  
Data Fusion ◽  
Object Detection ◽  
Sensor Fusion ◽  
Weather Conditions ◽  
Radar Data ◽  
Sensor Data ◽  
Data Set ◽  
3D Object ◽  
Individual Object ◽  
3D Object Detection

Automotive traffic scenes are complex due to the variety of possible scenarios, objects, and weather conditions that need to be handled. In contrast to more constrained environments, such as automated underground trains, automotive perception systems cannot be tailored to a narrow field of specific tasks but must handle an ever-changing environment with unforeseen events. As currently no single sensor is able to reliably perceive all relevant activity in the surroundings, sensor data fusion is applied to perceive as much information as possible. Data fusion of different sensors and sensor modalities on a low abstraction level enables the compensation of sensor weaknesses and misdetections among the sensors before the information-rich sensor data are compressed and thereby information is lost after a sensor-individual object detection. This paper develops a low-level sensor fusion network for 3D object detection, which fuses lidar, camera, and radar data. The fusion network is trained and evaluated on the nuScenes data set. On the test set, fusion of radar data increases the resulting AP (Average Precision) detection score by about 5.1% in comparison to the baseline lidar network. The radar sensor fusion proves especially beneficial in inclement conditions such as rain and night scenes. Fusing additional camera data contributes positively only in conjunction with the radar fusion, which shows that interdependencies of the sensors are important for the detection result. Additionally, the paper proposes a novel loss to handle the discontinuity of a simple yaw representation for object detection. Our updated loss increases the detection and orientation estimation performance for all sensor input configurations. The code for this research has been made available on GitHub.

scholarly journals Transfer Learning Based Semantic Segmentation for 3D Object Detection from Point Cloud

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 3964
Author(s):  
Muhammad Imad ◽  
Oualid Doukhi ◽  
Deok-Jin Lee
Keyword(s):  
Object Detection ◽  
Transfer Learning ◽  
Point Cloud ◽  
Large Scale ◽  
Semantic Segmentation ◽  
Point Cloud Data ◽  
Learning Stage ◽  
3D Object ◽  
Cloud Data ◽  
3D Object Detection

Three-dimensional object detection utilizing LiDAR point cloud data is an indispensable part of autonomous driving perception systems. Point cloud-based 3D object detection has been a better replacement for higher accuracy than cameras during nighttime. However, most LiDAR-based 3D object methods work in a supervised manner, which means their state-of-the-art performance relies heavily on a large-scale and well-labeled dataset, while these annotated datasets could be expensive to obtain and only accessible in the limited scenario. Transfer learning is a promising approach to reduce the large-scale training datasets requirement, but existing transfer learning object detectors are primarily for 2D object detection rather than 3D. In this work, we utilize the 3D point cloud data more effectively by representing the birds-eye-view (BEV) scene and propose a transfer learning based point cloud semantic segmentation for 3D object detection. The proposed model minimizes the need for large-scale training datasets and consequently reduces the training time. First, a preprocessing stage filters the raw point cloud data to a BEV map within a specific field of view. Second, the transfer learning stage uses knowledge from the previously learned classification task (with more data for training) and generalizes the semantic segmentation-based 2D object detection task. Finally, 2D detection results from the BEV image have been back-projected into 3D in the postprocessing stage. We verify results on two datasets: the KITTI 3D object detection dataset and the Ouster LiDAR-64 dataset, thus demonstrating that the proposed method is highly competitive in terms of mean average precision (mAP up to 70%) while still running at more than 30 frames per second (FPS).

scholarly journals 3D-FFS: Faster 3D object detection with Focused Frustum Search in sensor fusion based networks

2021 ◽  
Author(s):  
Aniruddha Ganguly ◽  
Tasin Ishmam ◽  
Khandker Aftarul Islam ◽  
Md Zahidur Rahman ◽  
Md. Shamsuzzoha Bayzid
Keyword(s):  
Object Detection ◽  
Sensor Fusion ◽  
3D Object ◽  
3D Object Detection

scholarly journals KDA3D: Key-Point Densification and Multi-Attention Guidance for 3D Object Detection

2020 ◽  
Vol 12 (11) ◽  
pp. 1895 ◽  
Author(s):  
Jiarong Wang ◽  
Ming Zhu ◽  
Bo Wang ◽  
Deyao Sun ◽  
Hua Wei ◽  
...  
Keyword(s):  
Object Detection ◽  
Network Architecture ◽  
Feature Learning ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Networks ◽  
3D Object ◽  
Rgb Images ◽  
Bounding Boxes ◽  
3D Object Detection

In this paper, we propose a novel 3D object detector KDA3D, which achieves high-precision and robust classification, segmentation, and localization with the help of key-point densification and multi-attention guidance. The proposed end-to-end neural network architecture takes LIDAR point clouds as the main inputs that can be optionally complemented by RGB images. It consists of three parts: part-1 segments 3D foreground points and generates reliable proposals; part-2 (optional) enhances point cloud density and reconstructs the more compact full-point feature map; part-3 refines 3D bounding boxes and adds semantic segmentation as extra supervision. Our designed lightweight point-wise and channel-wise attention modules can adaptively strengthen the “skeleton” and “distinctiveness” point-features to help feature learning networks capture more representative or finer patterns. The proposed key-point densification component can generate pseudo-point clouds containing target information from monocular images through the distance preference strategy and K-means clustering so as to balance the density distribution and enrich sparse features. Extensive experiments on the KITTI and nuScenes 3D object detection benchmarks show that our KDA3D produces state-of-the-art results while running in near real-time with a low memory footprint.

Sign in / Sign up

Export Citation Format

Share Document