Concurrent adjustment of active and passive optical sensors with GNSS and raw inertial data

This research focuses on augmenting the quality of digital mapping, notably from airborne sensors: camera and laser scanners. Progress is achieved by a joint combination of all critical observations in a common estimation scheme, called Dynamic Network (DN) or factor-graph. While earlier DN implementations treated imagery and lidar point-clouds independently, this thesis advances the methodology by integrating both datasets concurrently within a unified optimization framework. This research first investigates single-domain constraints, i.e. pixel-to-pixel (2D-2D) and point-to-point (3D-3D), in combination with satellite positioning (GNSS) and raw inertial observations. The methodology is then extended to cross-domain modeling by introducing a pixel-to-point (2D-3D) observation model, that enables direct coupling between images and point-clouds, starting with emulated data to establish theoretical observability. The extraction of 2Dâ 3D correspondences is realized first via domain-conversion techniques, i.e., 3D-3D links between dense image matching and lidar point-clouds or 2D-2D links between images and "rasterized" lidar clouds, and then via direct "learned methods". Results of single-domain constraints show considerable improvements in geo-referencing accuracy for inertial sensors of lower quality, as those used on smaller drones. The combined use of both 2D-2D and 3D-3D constraints improves the in-flight estimation of auxiliary parameters (as those related to system mounting - boresight), formally requiring specialized calibrations. Cross-domain conditioning shows great potential for further significant improvement of the geo-referencing accuracy when integrated with low-quality inertial sensors. Among the 2D-3D correspondence extraction techniques, those based on image and lidar clouds appear to be the most promising. The findings are supported by validations on data with modern sensors operated on a helicopter and an aircraft.