scholarly journals CoMask: Corresponding Mask-Based End-to-End Extrinsic Calibration of the Camera and LiDAR

2020 ◽  
Vol 12 (12) ◽  
pp. 1925
Author(s):  
Lu Yin ◽  
Bin Luo ◽  
Wei Wang ◽  
Huan Yu ◽  
Chenjie Wang ◽  
...  
Keyword(s):  
Optimization Problem ◽  
Autonomous Driving ◽  
Noise Resistance ◽  
Extrinsic Calibration ◽  
Manual Intervention ◽  
Extrinsic Parameters ◽  
Fully Automatic ◽  
End To End ◽  
Special Environment ◽  
3D Lidar

The integration of the camera and LiDAR has played an important role in the field of autonomous driving, for example in visual–LiDAR SLAM and 3D environment fusion perception, which rely on precise geometrical extrinsic calibration. In this paper, we proposed a fully automatic end-to-end method based on the 3D–2D corresponding mask (CoMask) to directly estimate the extrinsic parameters with high precision. Simple subtraction was applied to extract the candidate point cluster from the complex background, and then 3D LiDAR points located on checkerboard were selected and refined by spatial growth clustering. Once the distance transform of 2D checkerboard mask was generated, the extrinsic calibration of the two sensors could be converted to 3D–2D mask correspondence alignment. A simple but efficient strategy combining the genetic algorithm with the Levenberg–Marquardt method was used to solve the optimization problem globally without any initial estimates. Both simulated and realistic experiments showed that the proposed method could obtain accurate results without manual intervention, special environment setups, or prior initial parameters. Compared with the state of the art, our method has obvious advantages in accuracy, robustness, and noise resistance. Our code is open-source on GitHub.

scholarly journals Automatic Multi-Camera Extrinsic Parameter Calibration Based on Pedestrian Torsors †

Sensors ◽  
2019 ◽  
Vol 19 (22) ◽  
pp. 4989
Author(s):  
Truong ◽  
Philips ◽  
Deligiannis ◽  
Abrahamyan ◽  
Guan
Keyword(s):  
Brute Force ◽  
Camera Network ◽  
Extrinsic Calibration ◽  
Multiple Camera ◽  
Extrinsic Parameter ◽  
Extrinsic Parameters ◽  
Brute Force Method ◽  
Controlled Environments ◽  
Novel Method ◽  
Fully Automatic

Extrinsic camera calibration is essential for any computer vision task in a camera network. Typically, researchers place a calibration object in the scene to calibrate all the cameras in a camera network. However, when installing cameras in the field, this approach can be costly and impractical, especially when recalibration is needed. This paper proposes a novel, accurate and fully automatic extrinsic calibration framework for camera networks with partially overlapping views. The proposed method considers the pedestrians in the observed scene as the calibration objects and analyzes the pedestrian tracks to obtain extrinsic parameters. Compared to the state of the art, the new method is fully automatic and robust in various environments. Our method detect human poses in the camera images and then models walking persons as vertical sticks. We apply a brute-force method to determines the correspondence between persons in multiple camera images. This information along with 3D estimated locations of the top and the bottom of the pedestrians are then used to compute the extrinsic calibration matrices. We also propose a novel method to calibrate the camera network by only using the top and centerline of the person when the bottom of the person is not available in heavily occluded scenes. We verified the robustness of the method in different camera setups and for both single and multiple walking people. The results show that the triangulation error of a few centimeters can be obtained. Typically, it requires less than one minute of observing the walking people to reach this accuracy in controlled environments. It also just takes a few minutes to collect enough data for the calibration in uncontrolled environments. Our proposed method can perform well in various situations such as multi-person, occlusions, or even at real intersections on the street.

Imaging through Scattering Media with a Learning Based Prior

2020 ◽  
Vol 2020 (14) ◽  
pp. 306-1-306-6
Author(s):  
Florian Schiffers ◽  
Lionel Fiske ◽  
Pablo Ruiz ◽  
Aggelos K. Katsaggelos ◽  
Oliver Cossairt
Keyword(s):  
Optimization Problem ◽  
Autonomous Driving ◽  
Scattering Media ◽  
Photon Scattering ◽  
Point Spread ◽  
Spread Function ◽  
Radiative Transport Equation ◽  
Spatio Temporal ◽  
Transient Imaging

Imaging through scattering media finds applications in diverse fields from biomedicine to autonomous driving. However, interpreting the resulting images is difficult due to blur caused by the scattering of photons within the medium. Transient information, captured with fast temporal sensors, can be used to significantly improve the quality of images acquired in scattering conditions. Photon scattering, within a highly scattering media, is well modeled by the diffusion approximation of the Radiative Transport Equation (RTE). Its solution is easily derived which can be interpreted as a Spatio-Temporal Point Spread Function (STPSF). In this paper, we first discuss the properties of the ST-PSF and subsequently use this knowledge to simulate transient imaging through highly scattering media. We then propose a framework to invert the forward model, which assumes Poisson noise, to recover a noise-free, unblurred image by solving an optimization problem.

scholarly journals End-to-End Autonomous Driving Through Dueling Double Deep Q-Network

2021 ◽  
Author(s):  
Baiyu Peng ◽  
Qi Sun ◽  
Shengbo Eben Li ◽  
Dongsuk Kum ◽  
Yuming Yin ◽  
...  
Keyword(s):  
Neural Network ◽  
State Space ◽  
Learning Algorithm ◽  
Rapid Development ◽  
Autonomous Driving ◽  
Saliency Map ◽  
Hierarchical Architecture ◽  
Link Type ◽  
The Neural Network ◽  
End To End

AbstractRecent years have seen the rapid development of autonomous driving systems, which are typically designed in a hierarchical architecture or an end-to-end architecture. The hierarchical architecture is always complicated and hard to design, while the end-to-end architecture is more promising due to its simple structure. This paper puts forward an end-to-end autonomous driving method through a deep reinforcement learning algorithm Dueling Double Deep Q-Network, making it possible for the vehicle to learn end-to-end driving by itself. This paper firstly proposes an architecture for the end-to-end lane-keeping task. Unlike the traditional image-only state space, the presented state space is composed of both camera images and vehicle motion information. Then corresponding dueling neural network structure is introduced, which reduces the variance and improves sampling efficiency. Thirdly, the proposed method is applied to The Open Racing Car Simulator (TORCS) to demonstrate its great performance, where it surpasses human drivers. Finally, the saliency map of the neural network is visualized, which indicates the trained network drives by observing the lane lines. A video for the presented work is available online, https://youtu.be/76ciJmIHMD8 or https://v.youku.com/v_show/id_XNDM4ODc0MTM4NA==.html.

scholarly journals Extrinsic LiDAR/Ground Calibration Method Using 3D Geometrical Plane-Based Estimation

Sensors ◽  
2020 ◽  
Vol 20 (10) ◽  
pp. 2841
Author(s):  
Mohammad Ali Zaiter ◽  
Régis Lherbier ◽  
Ghaleb Faour ◽  
Oussama Bazzi ◽  
Jean-Charles Noyer
Keyword(s):  
Ground Plane ◽  
Ground Surface ◽  
Real Data ◽  
Calibration Method ◽  
Parameters Estimation ◽  
Parameters Optimization ◽  
Method Performance ◽  
Extrinsic Calibration ◽  
Extrinsic Parameters ◽  
The Stability

This paper details a new extrinsic calibration method for scanning laser rangefinder that is precisely focused on the geometrical ground plane-based estimation. This method is also efficient in the challenging experimental configuration of a high angle of inclination of the LiDAR. In this configuration, the calibration of the LiDAR sensor is a key problem that can be be found in various domains and in particular to guarantee the efficiency of ground surface object detection. The proposed extrinsic calibration method can be summarized by the following procedure steps: fitting ground plane, extrinsic parameters estimation (3D orientation angles and altitude), and extrinsic parameters optimization. Finally, the results are presented in terms of precision and robustness against the variation of LiDAR’s orientation and range accuracy, respectively, showing the stability and the accuracy of the proposed extrinsic calibration method, which was validated through numerical simulation and real data to prove the method performance.

scholarly journals End-to-End Game-Focused Learning of Adversary Behavior in Security Games

2020 ◽  
Vol 34 (02) ◽  
pp. 1378-1386
Author(s):  
Andrew Perrault ◽  
Bryan Wilder ◽  
Eric Ewing ◽  
Aditya Mate ◽  
Bistra Dilkina ◽  
...  
Keyword(s):  
Expected Utility ◽  
Optimization Problem ◽  
Predictive Accuracy ◽  
Experimental Results ◽  
Limited Data ◽  
Two Stage ◽  
Security Games ◽  
End To End ◽  
Feature Values ◽  
Adversary Model

Stackelberg security games are a critical tool for maximizing the utility of limited defense resources to protect important targets from an intelligent adversary. Motivated by green security, where the defender may only observe an adversary's response to defense on a limited set of targets, we study the problem of learning a defense that generalizes well to a new set of targets with novel feature values and combinations. Traditionally, this problem has been addressed via a two-stage approach where an adversary model is trained to maximize predictive accuracy without considering the defender's optimization problem. We develop an end-to-end game-focused approach, where the adversary model is trained to maximize a surrogate for the defender's expected utility. We show both in theory and experimental results that our game-focused approach achieves higher defender expected utility than the two-stage alternative when there is limited data.

Sign in / Sign up

Export Citation Format

Share Document