Home

Publication

Prompt–RSVQA: Prompting visual context to a language model for Remote Sensing Visual Question Answering

About
Privacy
Disclaimer

Graph Chatbot

Related publications (42)

Page 2 of 3

Advancing Self-Supervised Deep Learning for 3D Scene Understanding

Seyed Mohammad Mahdi Johari

Recent advancements in deep learning have revolutionized 3D computer vision, enabling the extraction of intricate 3D information from 2D images and video sequences. This thesis explores the application of deep learning in three crucial challenges of 3D com ...

EPFL2024

Toward universal cell embeddings: integrating single-cell RNA-seq datasets across species with SATURN

Maria Brbic

Analysis of single-cell datasets generated from diverse organisms offers unprecedented opportunities to unravel fundamental evolutionary processes of conservation and diversification of cell types. However, interspecies genomic differences limit the joint ...

Nature Publishing Group2024

Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modeling

Robert West, Maxime Jean Julien Peyrard, Marija Sakota

Generative language models (LMs) have become omnipresent across data science. For a wide variety of tasks, inputs can be phrased as natural language prompts for an LM, from whose output the solution can then be extracted. LM performance has consistently be ...

Assoc Computing Machinery2024

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Incorporating Projective Geometry into Deep Learning

Michal Jan Tyszkiewicz

In this thesis we explore the applications of projective geometry, a mathematical theory of the relation between 3D scenes and their 2D images, in modern learning-based computer vision systems. This is an interesting research question which contradicts the ...

EPFL2024

Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting

Haolin Chen

We are motivated primarily by the adaptation of text-to-speech synthesis models; however we argue that more generic parameter-efficient fine-tuning (PEFT) is an appropriate framework to do such adaptation. Nevertheless, catastrophic forgetting remains an i ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2024

Efficient local linearity regularization to overcome catastrophic overfitting

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

2024

Visual Saliency Prediction for Natural Images, Comics Panels, and Comics Pages

Bahar Aydemir

Recent years have seen remarkable advancements in saliency estimation methods, mainly due to deep learning models leveraging the widespread availability of real-world images. However, saliency is profoundly shaped by the intricacies of the human visual att ...

EPFL2024

Robust machine learning for neuroscientific inference

Steffen Schneider

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

EPFL2024

SVGC-AVA: 360-Degree Video Saliency Prediction With Spherical Vector-Based Graph Convolution and Audio-Visual Attention

Pascal Frossard

Viewers of 360-degree videos are provided with both visual modality to characterize their surrounding views and audio modality to indicate the sound direction. Though both modalities are important for saliency prediction, little work has been done by joint ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Visual complexity of urban streetscapes: human vs computer vision

Pietro Florio

Understanding visual complexity of urban environments may improve urban design strategies and limit visual pollution due to advertising, road signage, telecommunication systems and machinery. This paper aims at quantifying visual complexity specifically in ...

Springer2024

Serial Dependence in Human Visual Perception and Decision-Making

Gizay Ceylan

Decisions about a current visual stimulus are systematically biased by recently encountered stimuli, a phenomenon known as serial dependence. In human vision, for instance, we tend to report the features of current images as more similar â i.e., an attra ...

EPFL2024

Predicting Visual Stimuli From Cortical Response Recorded With Wide-Field Imaging in a Mouse

Silvestro Micera

Neural decoding of the visual system is a subject of research interest, both to understand how the visual system works and to be able to use this knowledge in areas, such as computer vision or brain-computer interfaces. Spike-based decoding is often used, ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Fast and Future: Towards Efficient Forecasting in Video Semantic Segmentation

Evann Pierre Guy Courdier

Deep learning has revolutionized the field of computer vision, a success largely attributable to the growing size of models, datasets, and computational power. Simultaneously, a critical pain point arises as several computer vision applications are deploye ...

EPFL2024

Fashioning Creative Expertise with Generative AI: Graphical Interfaces for GAN-Based Design Space Exploration Beter Support Ideation Than Text Prompts for Difusion Models

Pierre Dillenbourg, Tanja Christina Käser Jacober, Richard Lee Davis, Thiemo Wambsganss, Wei Jiang

This paper investigates the potential impact of deep generative models on the work of creative professionals. We argue that current generative modeling tools lack critical features that would make them useful creativity support tools, and introduce our own ...

Association for Computing Machinery2024

Text Representation Learning for Low Cost Natural Language Understanding

Jan Frederik Jonas Florian Mai

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

Can Generative AI Support Educators? Creating Learning Paths with PolyGloT

Tommaso Martorella

This chapter explores the intersection of generative AI and personalized learning, emphasizing the role of educators and examining the application of large language models (LLMs) in crafting personalized learning paths. The chapter examines the perception ...

Springer Nature (Switzerland)2024

Deep Generative Models for Autonomous Driving: from Motion Forecasting to Realistic Image Synthesis

Saeed Saadatnejad

Forecasting is a capability inherent in humans when navigating. Humans routinely plan their paths, considering the potential future movements of those around them. Similarly, to achieve comparable sophistication and safety, autonomous systems must embrace ...

EPFL2023

Land Cover Mapping From Multiple Complementary Experts Under Heavy Class Imbalance

Devis Tuia, Valérie Zermatten, Javiera Francisca Castillo Navarro

Deep learning has emerged as a promising avenue for automatic mapping, demonstrating high efficacy in land cover categorization through various semantic segmentation models. Nonetheless, the practical deployment of these models encounters important challen ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Improving Generalization of Pretrained Language Models

Rabeeh Karimi Mahabadi

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

EPFL2023