scholarly journals Robust Drivable Road Region Detection for Fixed-Route Autonomous Vehicles Using Map-Fusion Images

Sensors ◽  
2018 ◽  
Vol 18 (12) ◽  
pp. 4158 ◽  
Author(s):  
Yichao Cai ◽  
Dachuan Li ◽  
Xiao Zhou ◽  
Xingang Mou
Keyword(s):  
Neural Networks ◽  
Autonomous Vehicles ◽  
Autonomous Driving ◽  
Detection Accuracy ◽  
Region Detection ◽  
Detection Approach ◽  
Driving Scenario ◽  
Fusion Images ◽  
Environment Perception ◽  
Critical Heuristics

Environment perception is one of the major issues in autonomous driving systems. In particular, effective and robust drivable road region detection still remains a challenge to be addressed for autonomous vehicles in multi-lane roads, intersections and unstructured road environments. In this paper, a computer vision and neural networks-based drivable road region detection approach is proposed for fixed-route autonomous vehicles (e.g., shuttles, buses and other vehicles operating on fixed routes), using a vehicle-mounted camera, route map and real-time vehicle location. The key idea of the proposed approach is to fuse an image with its corresponding local route map to obtain the map-fusion image (MFI) where the information of the image and route map act as complementary to each other. The information of the image can be utilized in road regions with rich features, while local route map acts as critical heuristics that enable robust drivable road region detection in areas without clear lane marking or borders. A neural network model constructed upon the Convolutional Neural Networks (CNNs), namely FCN-VGG16, is utilized to extract the drivable road region from the fused MFI. The proposed approach is validated using real-world driving scenario videos captured by an industrial camera mounted on a testing vehicle. Experiments demonstrate that the proposed approach outperforms the conventional approach which uses non-fused images in terms of detection accuracy and robustness, and it achieves desirable robustness against undesirable illumination conditions and pavement appearance, as well as projection and map-fusion errors.

scholarly journals Road Characteristics Detection Based on Joint Convolutional Neural Networks with Adaptive Squares

2021 ◽  
Vol 10 (6) ◽  
pp. 377
Author(s):  
Chiao-Ling Kuo ◽  
Ming-Hua Tsai
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Autonomous Vehicles ◽  
Detection Accuracy ◽  
Geospatial Information ◽  
Combination Rules ◽  
And Performance ◽  
Road Characteristics ◽  
Machine Readable ◽  
Background Image

The importance of road characteristics has been highlighted, as road characteristics are fundamental structures established to support many transportation-relevant services. However, there is still huge room for improvement in terms of types and performance of road characteristics detection. With the advantage of geographically tiled maps with high update rates, remarkable accessibility, and increasing availability, this paper proposes a novel simple deep-learning-based approach, namely joint convolutional neural networks (CNNs) adopting adaptive squares with combination rules to detect road characteristics from roadmap tiles. The proposed joint CNNs are responsible for the foreground and background image classification and various types of road characteristics classification from previous foreground images, raising detection accuracy. The adaptive squares with combination rules help efficiently focus road characteristics, augmenting the ability to detect them and provide optimal detection results. Five types of road characteristics—crossroads, T-junctions, Y-junctions, corners, and curves—are exploited, and experimental results demonstrate successful outcomes with outstanding performance in reality. The information of exploited road characteristics with location and type is, thus, converted from human-readable to machine-readable, the results will benefit many applications like feature point reminders, road condition reports, or alert detection for users, drivers, and even autonomous vehicles. We believe this approach will also enable a new path for object detection and geospatial information extraction from valuable map tiles.

scholarly journals Automatic Roadway Features Detection with Oriented Object Detection

2021 ◽  
Vol 11 (8) ◽  
pp. 3531
Author(s):  
Hesham M. Eraqi ◽  
Karim Soliman ◽  
Dalia Said ◽  
Omar R. Elezaby ◽  
Mohamed N. Moustafa ◽  
...  
Keyword(s):  
Object Detection ◽  
Safety Evaluation ◽  
Autonomous Driving ◽  
Detection Accuracy ◽  
The Road ◽  
Detection Model ◽  
Detection Approach ◽  
Roadway Safety ◽  
Safety Features ◽  
Oriented Object

Extensive research efforts have been devoted to identify and improve roadway features that impact safety. Maintaining roadway safety features relies on costly manual operations of regular road surveying and data analysis. This paper introduces an automatic roadway safety features detection approach, which harnesses the potential of artificial intelligence (AI) computer vision to make the process more efficient and less costly. Given a front-facing camera and a global positioning system (GPS) sensor, the proposed system automatically evaluates ten roadway safety features. The system is composed of an oriented (or rotated) object detection model, which solves an orientation encoding discontinuity problem to improve detection accuracy, and a rule-based roadway safety evaluation module. To train and validate the proposed model, a fully-annotated dataset for roadway safety features extraction was collected covering 473 km of roads. The proposed method baseline results are found encouraging when compared to the state-of-the-art models. Different oriented object detection strategies are presented and discussed, and the developed model resulted in improving the mean average precision (mAP) by 16.9% when compared with the literature. The roadway safety feature average prediction accuracy is 84.39% and ranges between 91.11% and 63.12%. The introduced model can pervasively enable/disable autonomous driving (AD) based on safety features of the road; and empower connected vehicles (CV) to send and receive estimated safety features, alerting drivers about black spots or relatively less-safe segments or roads.

scholarly journals A Set of Single YOLO Modalities to Detect Occluded Entities via Viewpoint Conversion

2021 ◽  
Vol 11 (13) ◽  
pp. 6016
Author(s):  
Jinsoo Kim ◽  
Jeongho Cho
Keyword(s):  
Object Detection ◽  
Autonomous Vehicles ◽  
Autonomous Driving ◽  
Detection Algorithm ◽  
Detection Accuracy ◽  
Cloud Data ◽  
Detection Techniques ◽  
Bounding Boxes ◽  
Partially Occluded ◽  
Rgb Image

For autonomous vehicles, it is critical to be aware of the driving environment to avoid collisions and drive safely. The recent evolution of convolutional neural networks has contributed significantly to accelerating the development of object detection techniques that enable autonomous vehicles to handle rapid changes in various driving environments. However, collisions in an autonomous driving environment can still occur due to undetected obstacles and various perception problems, particularly occlusion. Thus, we propose a robust object detection algorithm for environments in which objects are truncated or occluded by employing RGB image and light detection and ranging (LiDAR) bird’s eye view (BEV) representations. This structure combines independent detection results obtained in parallel through “you only look once” networks using an RGB image and a height map converted from the BEV representations of LiDAR’s point cloud data (PCD). The region proposal of an object is determined via non-maximum suppression, which suppresses the bounding boxes of adjacent regions. A performance evaluation of the proposed scheme was performed using the KITTI vision benchmark suite dataset. The results demonstrate the detection accuracy in the case of integration of PCD BEV representations is superior to when only an RGB camera is used. In addition, robustness is improved by significantly enhancing detection accuracy even when the target objects are partially occluded when viewed from the front, which demonstrates that the proposed algorithm outperforms the conventional RGB-based model.

Obstacle Detection for Autonomous Driving Vehicles With Multi-LiDAR Sensor Fusion

2019 ◽  
Vol 142 (2) ◽  
Author(s):  
Mingcong Cao ◽  
Junmin Wang
Keyword(s):  
Data Processing ◽  
Autonomous Vehicles ◽  
Hybrid Electric Vehicle ◽  
Spatial Clustering ◽  
Autonomous Driving ◽  
Obstacle Detection ◽  
Lidar Data ◽  
Detection Accuracy ◽  
Clustering And Classification ◽  
Systematic Solution

Abstract In contrast to the single-light detection and ranging (LiDAR) system, multi-LiDAR sensors may improve the environmental perception for autonomous vehicles. However, an elaborated guideline of multi-LiDAR data processing is absent in the existing literature. This paper presents a systematic solution for multi-LiDAR data processing, which orderly includes calibration, filtering, clustering, and classification. As the accuracy of obstacle detection is fundamentally determined by noise filtering and object clustering, this paper proposes a novel filtering algorithm and an improved clustering method within the multi-LiDAR framework. To be specific, the applied filtering approach is based on occupancy rates (ORs) of sampling points. Besides, ORs are derived from the sparse “feature seeds” in each searching space. For clustering, the density-based spatial clustering of applications with noise (DBSCAN) is improved with an adaptive searching (AS) algorithm for higher detection accuracy. Besides, more robust and accurate obstacle detection can be achieved by combining AS-DBSCAN with the proposed OR-based filtering. An indoor perception test and an on-road test were conducted on a fully instrumented autonomous hybrid electric vehicle. Experimental results have verified the effectiveness of the proposed algorithms, which facilitate a reliable and applicable solution for obstacle detection.

Realtime On-Road Vehicle Detection Approach Based on Cascaded Structure

2011 ◽  
Vol 130-134 ◽  
pp. 2429-2432
Author(s):  
Liang Xiu Zhang ◽  
Xu Yun Qiu ◽  
Zhu Lin Zhang ◽  
Yu Lin Wang
Keyword(s):  
Warning System ◽  
Vehicle Detection ◽  
Autonomous Driving ◽  
Detection Algorithm ◽  
Detection Accuracy ◽  
Driver Assistance ◽  
Road Vehicle ◽  
Collision Warning ◽  
Detection Approach ◽  
Transportation Applications

Realtime on-road vehicle detection is a key technology in many transportation applications, such as driver assistance, autonomous driving and active safety. A vehicle detection algorithm based on cascaded structure is introduced. Haar-like features are used to built model in this application, and GAB algorithm is chosen to train the strong classifiers. Then, the real-time on-road vehicle classifier based on cascaded structure is constructed by combining the strong classifiers. Experimental results show that the cascaded classifier is excellent in both detection accuracy and computational efficiency, which ensures its application to collision warning system.

scholarly journals Surrounding Objects Detection and Tracking for Autonomous Driving Using LiDAR and Radar Fusion

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Ze Liu ◽  
Yingfeng Cai ◽  
Hai Wang ◽  
Long Chen
Keyword(s):  
High Precision ◽  
Autonomous Vehicles ◽  
Unscented Kalman Filter ◽  
Autonomous Vehicle ◽  
Autonomous Driving ◽  
Detection Accuracy ◽  
Environmental Sensors ◽  
Detection And Tracking ◽  
Vehicle Test ◽  
Objects Detection

AbstractRadar and LiDAR are two environmental sensors commonly used in autonomous vehicles, Lidars are accurate in determining objects’ positions but significantly less accurate as Radars on measuring their velocities. However, Radars relative to Lidars are more accurate on measuring objects velocities but less accurate on determining their positions as they have a lower spatial resolution. In order to compensate for the low detection accuracy, incomplete target attributes and poor environmental adaptability of single sensors such as Radar and LiDAR, in this paper, an effective method for high-precision detection and tracking of surrounding targets of autonomous vehicles. By employing the Unscented Kalman Filter, Radar and LiDAR information is effectively fused to achieve high-precision detection of the position and speed information of targets around the autonomous vehicle. Finally, the real vehicle test under various driving environment scenarios is carried out. The experimental results show that the proposed sensor fusion method can effectively detect and track the vehicle peripheral targets with high accuracy. Compared with a single sensor, it has obvious advantages and can improve the intelligence level of autonomous cars.

scholarly journals Policy-Gradient and Actor-Critic Based State Representation Learning for Safe Driving of Autonomous Vehicles

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 5991 ◽  
Author(s):  
Abhishek Gupta ◽  
Ahmed Shaharyar Khwaja ◽  
Alagan Anpalagan ◽  
Ling Guan ◽  
Bala Venkatesh
Keyword(s):  
Autonomous Vehicles ◽  
Representation Learning ◽  
Autonomous Driving ◽  
State Representation ◽  
Learning Loss ◽  
Safe Driving ◽  
Reward Function ◽  
Value Loss ◽  
Policy Gradient ◽  
Environment Perception

In this paper, we propose an environment perception framework for autonomous driving using state representation learning (SRL). Unlike existing Q-learning based methods for efficient environment perception and object detection, our proposed method takes the learning loss into account under deterministic as well as stochastic policy gradient. Through a combination of variational autoencoder (VAE), deep deterministic policy gradient (DDPG), and soft actor-critic (SAC), we focus on uninterrupted and reasonably safe autonomous driving without steering off the track for a considerable driving distance. Our proposed technique exhibits learning in autonomous vehicles under complex interactions with the environment, without being explicitly trained on driving datasets. To ensure the effectiveness of the scheme over a sustained period of time, we employ a reward-penalty based system where a negative reward is associated with an unfavourable action and a positive reward is awarded for favourable actions. The results obtained through simulations on DonKey simulator show the effectiveness of our proposed method by examining the variations in policy loss, value loss, reward function, and cumulative reward for ‘VAE+DDPG’ and ‘VAE+SAC’ over the learning process.

scholarly journals Real-Time Hybrid Multi-Sensor Fusion Framework for Perception in Autonomous Vehicles

Sensors ◽  
2019 ◽  
Vol 19 (20) ◽  
pp. 4357 ◽  
Author(s):  
Babak Shahian Jahromi ◽  
Theja Tulabandhula ◽  
Sabri Cetin
Keyword(s):  
Real Time ◽  
Sensor Fusion ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Obstacle Detection ◽  
Sensor Data ◽  
Detection Accuracy ◽  
System Solution ◽  
Environment Perception ◽  
Fusion Framework

There are many sensor fusion frameworks proposed in the literature using different sensors and fusion methods combinations and configurations. More focus has been on improving the accuracy performance; however, the implementation feasibility of these frameworks in an autonomous vehicle is less explored. Some fusion architectures can perform very well in lab conditions using powerful computational resources; however, in real-world applications, they cannot be implemented in an embedded edge computer due to their high cost and computational need. We propose a new hybrid multi-sensor fusion pipeline configuration that performs environment perception for autonomous vehicles such as road segmentation, obstacle detection, and tracking. This fusion framework uses a proposed encoder-decoder based Fully Convolutional Neural Network (FCNx) and a traditional Extended Kalman Filter (EKF) nonlinear state estimator method. It also uses a configuration of camera, LiDAR, and radar sensors that are best suited for each fusion method. The goal of this hybrid framework is to provide a cost-effective, lightweight, modular, and robust (in case of a sensor failure) fusion system solution. It uses FCNx algorithm that improve road detection accuracy compared to benchmark models while maintaining real-time efficiency that can be used in an autonomous vehicle embedded computer. Tested on over 3K road scenes, our fusion algorithm shows better performance in various environment scenarios compared to baseline benchmark networks. Moreover, the algorithm is implemented in a vehicle and tested using actual sensor data collected from a vehicle, performing real-time environment perception.

scholarly journals CAEV-Surrounding Objects Detection and Tracking for Autonomous Driving Using Lidar and Radar Fusion

2020 ◽  
Author(s):  
Ze Liu ◽  
Feng Ying Cai
Keyword(s):  
High Precision ◽  
Autonomous Vehicles ◽  
Unscented Kalman Filter ◽  
Autonomous Vehicle ◽  
Autonomous Driving ◽  
Detection Accuracy ◽  
Environmental Sensors ◽  
Detection And Tracking ◽  
Single Sensor ◽  
Objects Detection

Abstract Radar and Lidar are two environmental sensors commonly used in autonomous vehicles，Lidars are accurate in determining objects’ positions but significantly less accurate on measuring their velocities. However, Radars are more accurate on measuring objects velocities but less accurate on determining their positions as they have a lower spatial resolution. In order to compensate for the low detection accuracy, incomplete target attributes and poor environmental adaptability of single sensors such as Radar and LIDAR, we proposed an effective method for high-precision detection and tracking of surrounding targets of autonomous vehicles. By employing the Unscented Kalman Filter, radar and LIDAR information is effectively fused to achieve high-precision detection of the position and speed information of targets around the autonomous vehicle. Finally, we do a variety of driving environment under the real car algorithm verification test. The experimental results show that the proposed sensor fusion method can effectively detect and track the vehicle peripheral targets with high accuracy. Compared with a single sensor, it has obvious advantages and can improve the intelligence level of driverless cars.

Challenges and Requirements of Immersive Media in Autonomous Car: Exploring the Feasibility of Virtual Entertainment Applications

i-com ◽  
2019 ◽  
Vol 18 (2) ◽  
pp. 105-125
Author(s):  
Carolin Wienrich ◽  
Kristina Schindler
Keyword(s):  
User Experience ◽  
Autonomous Vehicles ◽  
Driving Simulator ◽  
Autonomous Driving ◽  
Simulator Sickness ◽  
Significant Difference ◽  
Driving Scenario ◽  
Immersive Media

Abstract This paper investigated the influence of VR-entertainment systems on passenger and entertainment experience in vehicles with smooth movements. To simulate an autonomous driving scenario, a tablet and a mobile VR-HMD were evaluated in a dynamic driving simulator. Passenger, user and entertainment experience were measured through questionnaires based on comfort/discomfort, application perception, presence, and simulator sickness. In two experiments, two film sequences with varying formats (2D versus 3D) were presented. In Experiment 1, the established entertainment system (tablet + 2D) was tested against a possible future one (HMD + 3D). The results indicated a significantly more favorable experience for the VR-HMD application in the dimensions of user experience (UX) and presence, as well as low simulator sickness values. In Experiment 2, the film format was held constant (2D), and only the device (tablet versus HMD) was varied. There was a significant difference in all constructs, which points to a positive reception of the HMD. Additional analyses of the HMD device data for both experiments showed that the device and not the film format contributed to the favorable experience with the HMD. Additionally, the framework to evaluate the new application context of VR as an entertainment system in autonomous vehicles was discussed.

Sign in / Sign up

Export Citation Format

Share Document