Extraction of Image-to-Lidar Correspondences and their Impact on Optimal Sensor Fusion

Abstract. This work extends our initial proof-of-concept via emulations on the benefits of relative spatial constraints between imagery and lidar point clouds in a factor graph based optimization with satellite positioning (GNSS) and raw inertial readings (Mouzakidou et al., 2025). Here, we demonstrate practically the automatic extraction and integration of 2D-3D correspondences established in the 3D domain within rough natural terrain flown over by an aircraft with sensors of high quality. We show that considering cross-domain (i.e. 2D-3D) constraints enables the calibration of internal camera parameters and its boresight on job, i.e. within mapping flight configurations, where conventional approaches fail. The common optimization of raw IMU data with such constraints improves the respective agreements between the lidar and image dense clouds, achieving consistency at ground resolution level, which is not the case for the conventional (standard) processing of acquired data.