Multimodal Land Cover Mapping from Remote Sensing Imagery, Species Observations, and Language

Land cover change is widely recognised as a major driver of biodiversity loss for terrestrial ecosystems, making accurate characterisation of Earth's surface essential for nature conservation. Progress in this domain increasingly relies on integrating multiple data sources, such as satellite imagery, camera traps, in situ observations, text documentation, each providing a complementary view of natural processes. Yet such integration remains challenging: each modality differs fundamentally in information content, format, noise levels, and geographic coverage. This thesis addresses these challenges by proposing methods to combine these rich and heterogeneous data to better characterise our natural environment. We leverage recent advances in machine learning, specifically deep neural networks and multimodal learning methods. We integrate remote sensing imagery with other sources of information, such as species occurrence records, environmental covariates, elevation data and text data. In particular, we demonstrate that language integration improves interaction with these data and enables a better understanding of our natural world. This thesis makes four contributions to this goal. First, we propose a multi-expert model that improves recognition of rare land cover categories in alpine areas from remote sensing imagery and elevation data. Second, we present TACOSS, which proposes to interact with remote sensing images through natural language and is able to recognise land cover categories across different classification frameworks and nomenclatures. Third, we introduce EcoWikiRS, which learns ecologically informed remote sensing image representations by leveraging GBIF species observations and Wikipedia habitat descriptions as supervision, yielding improved recognition of natural habitats. Fourth, we propose GAMMA, a unifying approach to combine sparse and continuous geodata sources, such as geolocated text and remote sensing imagery, by explicitly incorporating geolocation information. Together, these contributions demonstrate that multimodal approaches unlock new ways of integrating language into Earth observation workflows: broadening access to remote sensing data, improving the transferability of knowledge between nomenclatures, and enabling the integration of knowledge from textual resources. Finally, we reflect on persistent challenges in the field, providing perspectives on the contribution of individual modalities within multimodal pipelines, and considerations on safety and sustainability in the context of AI-based environmental monitoring.