GEOCOMP demos

Automatic targetless Lidar-camera calibration

Sensor fusion is one of the main challenges in self driving and robotics applications. In this paper we propose an automatic, online and target-less camera-Lidar extrinsic calibration approach. We adopt a structure from motion (SfM) method to generate 3D point clouds from the camera data which can be matched to the Lidar point clouds; thus, we address the extrinsic calibration problem as a registration task in the 3D domain. The core step of the approach is a two-stage transformation estimation: First, we introduce an object level coarse alignment algorithm operating in the Hough space to transform the SfM-based and the Lidar point clouds into a common coordinate system. Thereafter, we apply a control point based nonrigid transformation refinement step to register the point clouds more precisely. Finally, we calculate the correspondences between the 3D Lidar points and the pixels in the 2D camera domain. We evaluated the method in various real-life traffic scenarios in Budapest, Hungary. The results show that our proposed extrinsic calibration approach is able to provide accurate and robust parameter settings on-the-fly.

 

Worklow of the proposed automatic Lidar-camera calibration approach [1]

 

Demonstation of the calibration results:

Camera-lidar fusion and multi-object tracking from a moving platform equipped with 4 cameras and a Velodyne HDL64 sensor @ SZTAKI

Demonstration of Lidar-only object tracking from a moving platform:

Object tracking in Velodyne HDL-64 point clouds @ SZTAKI

 

References

[1]B. Nagy and Cs. Benedek: ”On-the-Fly Camera and Lidar Calibration,” Remote Sensing, vol 12, no. 7, article 1137, 2020, IF: 4.848 Open Access

[2]B. Nagy, L.A. Kovács and Cs. Benedek: "SFM and Semantic Information Based Online Targetless Camera-Lidar Self-Calibration", IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 22-25 September, 2019

[3]B. Nagy, L.A. Kovács and Cs. Benedek: "Online Targetless End-to-End Camera-LIDAR Self-calibration", International Conference on Machine Vision Applications (MVA), Tokyo, Japan, 27-31 May, 2019