scholarly journals Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

Electronics ◽  
2020 ◽  
Vol 9 (3) ◽  
pp. 451 ◽  
Author(s):  
Limin Guan ◽  
Yi Chen ◽  
Guiping Wang ◽  
Xu Lei
Keyword(s):  
Real Time ◽  
Color Image ◽  
Evidence Theory ◽  
Depth Map ◽  
Vehicle Detection ◽  
Detection Accuracy ◽  
Distance Information ◽  
Detection Model ◽  
Detection Mode ◽  
Single Sensor

Vehicle detection is essential for driverless systems. However, the current single sensor detection mode is no longer sufficient in complex and changing traffic environments. Therefore, this paper combines camera and light detection and ranging (LiDAR) to build a vehicle-detection framework that has the characteristics of multi adaptability, high real-time capacity, and robustness. First, a multi-adaptive high-precision depth-completion method was proposed to convert the 2D LiDAR sparse depth map into a dense depth map, so that the two sensors are aligned with each other at the data level. Then, the You Only Look Once Version 3 (YOLOv3) real-time object detection model was used to detect the color image and the dense depth map. Finally, a decision-level fusion method based on bounding box fusion and improved Dempster–Shafer (D–S) evidence theory was proposed to merge the two results of the previous step and obtain the final vehicle position and distance information, which not only improves the detection accuracy but also improves the robustness of the whole framework. We evaluated our method using the KITTI dataset and the Waymo Open Dataset, and the results show the effectiveness of the proposed depth completion method and multi-sensor fusion strategy.

scholarly journals Research on Lightweight Infrared Pedestrian Detection Model Algorithm for Embedded Platform

2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Zhaoli Wu ◽  
Xin Wang ◽  
Chao Chen
Keyword(s):  
Real Time ◽  
Target Detection ◽  
Pedestrian Detection ◽  
Infrared Image ◽  
Far Infrared ◽  
Detection Algorithm ◽  
Model Parameters ◽  
Detection Accuracy ◽  
Detection Model ◽  
Embedded Platform

Due to the limitation of energy consumption and power consumption, the embedded platform cannot meet the real-time requirements of the far-infrared image pedestrian detection algorithm. To solve this problem, this paper proposes a new real-time infrared pedestrian detection algorithm (RepVGG-YOLOv4, Rep-YOLO), which uses RepVGG to reconstruct the YOLOv4 backbone network, reduces the amount of model parameters and calculations, and improves the speed of target detection; using space spatial pyramid pooling (SPP) obtains different receptive field information to improve the accuracy of model detection; using the channel pruning compression method reduces redundant parameters, model size, and computational complexity. The experimental results show that compared with the YOLOv4 target detection algorithm, the Rep-YOLO algorithm reduces the model volume by 90%, the floating-point calculation is reduced by 93.4%, the reasoning speed is increased by 4 times, and the model detection accuracy after compression reaches 93.25%.

scholarly journals Real-Time Vehicle Detection Algorithm Based on Vision and Lidar Point Cloud Fusion

2019 ◽  
Vol 2019 ◽  
pp. 1-9 ◽  
Author(s):  
Hai Wang ◽  
Xinyu Lou ◽  
Yingfeng Cai ◽  
Yicheng Li ◽  
Long Chen
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Autonomous Vehicles ◽  
Point Cloud ◽  
Vehicle Detection ◽  
Detection Algorithm ◽  
Detection Methods ◽  
Depth Information ◽  
Detection Accuracy ◽  
Classification Rate

Vehicle detection is one of the most important environment perception tasks for autonomous vehicles. The traditional vision-based vehicle detection methods are not accurate enough especially for small and occluded targets, while the light detection and ranging- (lidar-) based methods are good in detecting obstacles but they are time-consuming and have a low classification rate for different target types. Focusing on these shortcomings to make the full use of the advantages of the depth information of lidar and the obstacle classification ability of vision, this work proposes a real-time vehicle detection algorithm which fuses vision and lidar point cloud information. Firstly, the obstacles are detected by the grid projection method using the lidar point cloud information. Then, the obstacles are mapped to the image to get several separated regions of interest (ROIs). After that, the ROIs are expanded based on the dynamic threshold and merged to generate the final ROI. Finally, a deep learning method named You Only Look Once (YOLO) is applied on the ROI to detect vehicles. The experimental results on the KITTI dataset demonstrate that the proposed algorithm has high detection accuracy and good real-time performance. Compared with the detection method based only on the YOLO deep learning, the mean average precision (mAP) is increased by 17%.

scholarly journals Vehicle Detection and Ranging Using Two Different Focal Length Cameras

2020 ◽  
Vol 2020 ◽  
pp. 1-14 ◽  
Author(s):  
Jun Liu ◽  
Rui Zhang
Keyword(s):  
Real Time ◽  
Focal Length ◽  
Vehicle Detection ◽  
Autonomous Driving ◽  
License Plate ◽  
The Road ◽  
Detection Model ◽  
Automatic Driving ◽  
Convolution Structure ◽  
Model Size

Vehicle detection is a crucial task for autonomous driving and demands high accuracy and real-time speed. Considering that the current deep learning object detection model size is too large to be deployed on the vehicle, this paper introduces the lightweight network to modify the feature extraction layer of YOLOv3 and improve the remaining convolution structure, and the improved Lightweight YOLO network reduces the number of network parameters to a quarter. Then, the license plate is detected to calculate the actual vehicle width and the distance between the vehicles is estimated by the width. This paper proposes a detection and ranging fusion method based on two different focal length cameras to solve the problem of difficult detection and low accuracy caused by a small license plate when the distance is far away. The experimental results show that the average precision and recall of the Lightweight YOLO trained on the self-built dataset is 4.43% and 3.54% lower than YOLOv3, respectively, but the computing speed of the network decreases 49 ms per frame. The road experiments in different scenes also show that the long and short focal length camera fusion ranging method dramatically improves the accuracy and stability of ranging. The mean error of ranging results is less than 4%, and the range of stable ranging can reach 100 m. The proposed method can realize real-time vehicle detection and ranging on the on-board embedded platform Jetson Xavier, which satisfies the requirements of automatic driving environment perception.

scholarly journals Real-Time Vehicle-Detection Method in Bird-View Unmanned-Aerial-Vehicle Imagery

Sensors ◽  
2019 ◽  
Vol 19 (18) ◽  
pp. 3958
Author(s):  
Seongkyun Han ◽  
Jisang Yoo ◽  
Soonchul Kwon
Keyword(s):  
Real Time ◽  
Unmanned Aerial Vehicle ◽  
Detection Method ◽  
Vehicle Detection ◽  
Research Area ◽  
Angular Distortion ◽  
Background Information ◽  
Feature Maps ◽  
Detection Model ◽  
Aerial Vehicle

Vehicle detection is an important research area that provides background information for the diversity of unmanned-aerial-vehicle (UAV) applications. In this paper, we propose a vehicle-detection method using a convolutional-neural-network (CNN)-based object detector. We design our method, DRFBNet300, with a Deeper Receptive Field Block (DRFB) module that enhances the expressiveness of feature maps to detect small objects in the UAV imagery. We also propose the UAV-cars dataset that includes the composition and angular distortion of vehicles in UAV imagery to train our DRFBNet300. Lastly, we propose a Split Image Processing (SIP) method to improve the accuracy of the detection model. Our DRFBNet300 achieves 21 mAP with 45 FPS in the MS COCO metric, which is the highest score compared to other lightweight single-stage methods running in real time. In addition, DRFBNet300, trained on the UAV-cars dataset, obtains the highest AP score at altitudes of 20–50 m. The gap of accuracy improvement by applying the SIP method became larger when the altitude increases. The DRFBNet300 trained on the UAV-cars dataset with SIP method operates at 33 FPS, enabling real-time vehicle detection.

scholarly journals High-Accuracy Real-Time Fish Detection Based on Self-Build Dataset and RIRD-YOLOv3

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Wenkai Wang ◽  
Bingwei He ◽  
Liwei Zhang
Keyword(s):  
Real Time ◽  
High Accuracy ◽  
Frame Rate ◽  
Detection Accuracy ◽  
Average Precision ◽  
Recovery Algorithm ◽  
Detection Model ◽  
Real Time Image ◽  
Fish Detection ◽  
Time Image

To better detect fish in an aquaculture environment, a high-accuracy real-time detection model is proposed. An experimental dataset was collected for fish detection in laboratory aquaculture environments using remotely operated vehicles. To overcome the inaccuracy of the You Only Look Once v3 (YOLOv3) algorithm in underwater farming environment, a suitable set of hyperparameters was obtained through multiple sets of experiments. Then, a real-time image recovery algorithm is applied before YOLOv3 to reduce the effects of both noise and light on images whilst keeping the real-time capability, leading to a mean average precision of 0.85 and frame rate of 17.6 fps, respectively. Finally, compared with the base detection model using only the YOLOv3 algorithm, the enhanced detection model presented results in a reduction of miss detection rate from 23% to only 9% across different environments and with the detection accuracy of the target in different environments being improved from 8% to 37%.

scholarly journals An Improved YOLOv2 for Vehicle Detection

Sensors ◽  
2018 ◽  
Vol 18 (12) ◽  
pp. 4272 ◽  
Author(s):  
Jun Sang ◽  
Zhongyuan Wu ◽  
Pei Guo ◽  
Haibo Hu ◽  
Hong Xiang ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Vehicle Detection ◽  
Training Dataset ◽  
Validation Dataset ◽  
Detection Accuracy ◽  
Extraction Ability ◽  
Detection Model ◽  
Vehicle Type ◽  
Proposed Model ◽  
Bounding Boxes

Vehicle detection is one of the important applications of object detection in intelligent transportation systems. It aims to extract specific vehicle-type information from pictures or videos containing vehicles. To solve the problems of existing vehicle detection, such as the lack of vehicle-type recognition, low detection accuracy, and slow speed, a new vehicle detection model YOLOv2_Vehicle based on YOLOv2 is proposed in this paper. The k-means++ clustering algorithm was used to cluster the vehicle bounding boxes on the training dataset, and six anchor boxes with different sizes were selected. Considering that the different scales of the vehicles may influence the vehicle detection model, normalization was applied to improve the loss calculation method for length and width of bounding boxes. To improve the feature extraction ability of the network, the multi-layer feature fusion strategy was adopted, and the repeated convolution layers in high layers were removed. The experimental results on the Beijing Institute of Technology (BIT)-Vehicle validation dataset demonstrated that the mean Average Precision (mAP) could reach 94.78%. The proposed model also showed excellent generalization ability on the CompCars test dataset, where the “vehicle face” is quite different from the training dataset. With the comparison experiments, it was proven that the proposed method is effective for vehicle detection. In addition, with network visualization, the proposed model showed excellent feature extraction ability.

scholarly journals Real-time RGB-D image stitching using multiple Kinects for improved field of view

2017 ◽  
Vol 14 (2) ◽  
pp. 172988141769556 ◽  
Author(s):  
Hengyu Li ◽  
Hang Liu ◽  
Ning Cao ◽  
Yan Peng ◽  
Shaorong Xie ◽  
...  
Keyword(s):  
Real Time ◽  
Anisotropic Diffusion ◽  
Color Image ◽  
Depth Map ◽  
Color Images ◽  
Field Of View ◽  
Hole Filling ◽  
Depth Data ◽  
Registration Data ◽  
Filling Method

This article concerns the problems of a defective depth map and limited field of view of Kinect-style RGB-D sensors. An anisotropic diffusion based hole-filling method is proposed to recover invalid depth data in the depth map. The field of view of the Kinect-style RGB-D sensor is extended by stitching depth and color images from several RGB-D sensors. By aligning the depth map with the color image, the registration data calculated by registering color images can be used to stitch depth and color images into a depth and color panoramic image concurrently in real time. Experiments show that the proposed stitching method can generate a RGB-D panorama with no invalid depth data and little distortion in real time and can be extended to incorporate more RGB-D sensors to construct even a 360° field of view panoramic RGB-D image.

scholarly journals Development of Monitoring Robot System for Tomato Fruits in Hydroponic Greenhouses

Agronomy ◽  
2021 ◽  
Vol 11 (11) ◽  
pp. 2211
Author(s):  
Dasom Seo ◽  
Byeong-Hyo Cho ◽  
Kyoungchul Kim
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Learning Algorithms ◽  
Detection Accuracy ◽  
Tomato Fruits ◽  
Agricultural Robots ◽  
Crop Monitoring ◽  
Detection Model ◽  
Stable Performance ◽  
Expert Classification

Crop monitoring is highly important in terms of the efficient and stable performance of tasks such as planting, spraying, and harvesting, and for this reason, several studies are being conducted to develop and improve crop monitoring robots. In addition, the applications of deep learning algorithms are increasing in the development of agricultural robots since deep learning algorithms that use convolutional neural networks have been proven to show outstanding performance in image classification, segmentation, and object detection. However, most of these applications are focused on the development of harvesting robots, and thus, there are only a few studies that improve and develop monitoring robots through the use of deep learning. For this reason, we aimed to develop a real-time robot monitoring system for the generative growth of tomatoes. The presented method detects tomato fruits grown in hydroponic greenhouses using the Faster R-CNN (region-based convolutional neural network). In addition, we sought to select a color model that was robust to external light, and we used hue values to develop an image-based maturity standard for tomato fruits; furthermore, the developed maturity standard was verified through comparison with expert classification. Finally, the number of tomatoes was counted using a centroid-based tracking algorithm. We trained the detection model using an open dataset and tested the whole system in real-time in a hydroponic greenhouse. A total of 53 tomato fruits were used to verify the developed system, and the developed system achieved 88.6% detection accuracy when completely obscured fruits not captured by the camera were included. When excluding obscured fruits, the system’s accuracy was 90.2%. For the maturity classification, we conducted qualitative evaluations with the assistance of experts.

scholarly journals Embedded YOLO: A Real-Time Object Detector for Small Intelligent Trajectory Cars

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
WenYu Feng ◽  
YuanFan Zhu ◽  
JunTai Zheng ◽  
Han Wang
Keyword(s):  
Object Detection ◽  
Real Time ◽  
Network Structure ◽  
Lightweight Design ◽  
Detection Accuracy ◽  
Embedded Computing ◽  
Embedded Devices ◽  
Detection Model ◽  
Parallel Acceleration ◽  
Accuracy Performance

YOLO-Tiny is a lightweight version of the object detection model based on the original “You only look once” (YOLO) model for simplifying network structure and reducing parameters, which makes it suitable for real-time applications. Although the YOLO-Tiny series, which includes YOLOv3-Tiny and YOLOv4-Tiny, can achieve real-time performance on a powerful GPU, it remains challenging to leverage this approach for real-time object detection on embedded computing devices, such as those in small intelligent trajectory cars. To obtain real-time and high-accuracy performance on these embedded devices, a novel object detection lightweight network called embedded YOLO is proposed in this paper. First, a new backbone network structure, ASU-SPP network, is proposed to enhance the effectiveness of low-level features. Then, we designed a simplified version of the neck network module PANet-Tiny that reduces computation complexity. Finally, in the detection head module, we use depthwise separable convolution to reduce the number of convolution stacks. In addition, the number of channels is reduced to 96 dimensions so that the module can attain the parallel acceleration of most inference frameworks. With its lightweight design, the proposed embedded YOLO model has only 3.53M parameters, and the average processing time can reach 155.1 frames per second, as verified by Baidu smart car target detection. At the same time, compared with YOLOv3-Tiny and YOLOv4-Tiny, the detection accuracy is 6% higher.

scholarly journals YOLO-Highway: An Improved Highway Center Marking Detection Model for Unmanned Aerial Vehicle Autonomous Flight

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Zhiwei Zhao ◽  
Jianfeng Han ◽  
Lili Song
Keyword(s):  
Real Time ◽  
Unmanned Aerial Vehicle ◽  
Visual Navigation ◽  
Detection Accuracy ◽  
Highway Maintenance ◽  
The Real ◽  
Detection Model ◽  
Proposed Model ◽  
Time Requirements ◽  
Aerial Vehicle

Automatic visual navigation flight of an unmanned aerial vehicle (UAV) plays an important role in the highway maintenance field. Automatic highway center marking detection is the most important part of the visual navigation flight of a UAV. In this study, the UAV-viewed highway data are collected from the UAV perspective. This paper proposes a model named the YOLO-Highway that uses an improved form of the You Only Look Once (YOLO) model to enhance the real-time detection of highway marking problems. The proposed model is mainly designed by optimizing the network structure and the loss function of the original YOLOv3 model. The proposed model is verified by the experiments using the highway center marking dataset, and the results show that the average precision (AP) of the trained model is 82.79%, and the frames per second (FPS) is 25.71 f/s. In comparison with the original YOLOv3 model, the detection accuracy of the proposed model is improved by 7.05%, and its speed is improved by 5.29 f/s. Moreover, the proposed model had stronger environmental adaptability and better detection precision and speed than the original model in complex highway scenarios. The experimental results show that the proposed YOLO-Highway model can accurately detect the highway center markings in real-time and has high robustness to changes in different environmental conditions. Therefore, the YOLO-Highway model can meet the real-time requirements of the highway center marking detection.

Sign in / Sign up

Export Citation Format

Share Document