Publications related to Audio-Visual Fusion : New Methods and Applications

Related publications (31)

Aggregating Spatial and Photometric Context for Photometric Stereo

Photometric stereo, a computer vision technique for estimating the 3D shape of objects through images captured under varying illumination conditions, has been a topic of research for nearly four decades. In its general formulation, photometric stereo is an ...

EPFL2024

Fast and Future: Towards Efficient Forecasting in Video Semantic Segmentation

Evann Pierre Guy Courdier

Deep learning has revolutionized the field of computer vision, a success largely attributable to the growing size of models, datasets, and computational power. Simultaneously, a critical pain point arises as several computer vision applications are deploye ...

EPFL2024

Model reduction of coupled systems based on non-intrusive approximations of the boundary response maps

Jan Sickmann Hesthaven, Niccolo' Discacciati

We propose a local, non -intrusive model order reduction technique to accurately approximate the solution of coupled multi -component parametrized systems governed by partial differential equations. Our approach is based on the approximation of the boundar ...

Elsevier Science Sa2024

Efficient Temporally-Aware DeepFake Detection using H.264 Motion Vectors

Peter Arpad Grönquist, Sabine Süsstrunk, Yufan Ren

Video DeepFakes are fake media created with Deep Learning (DL) that manipulate a person’s expression or identity. Most current DeepFake detection methods analyze each frame independently, ignoring inconsistencies and unnatural movements between frames. Som ...

2024

A Gradient-Gated SPAD Array for Non-Line-of-Sight Imaging

Edoardo Charbon, Francesco Gramuglia, Myung Jae Lee, Pouyan Keshavarzian, Jiuxuan Zhao

Time-resolved non-line-of-sight (NLOS) imaging based on single-photon avalanche diode (SPAD) detectors have demonstrated impressive results in recent years. To acquire adequate number of indirect photons from a hidden scene in the presence of overwhelming ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Subjective performance evaluation of bitrate allocation strategies for MPEG and JPEG Pleno point cloud compression

Touradj Ebrahimi, Michela Testolina, Davi Nachtigall Lazzarotto

The recent rise in interest in point clouds as an imaging modality has motivated standardization groups such as JPEG and MPEG to launch activities aiming at developing compression standards for point clouds. Lossy compression usually introduces visual arti ...

Springer2024

SVGC-AVA: 360-Degree Video Saliency Prediction With Spherical Vector-Based Graph Convolution and Audio-Visual Attention

Pascal Frossard

Viewers of 360-degree videos are provided with both visual modality to characterize their surrounding views and audio modality to indicate the sound direction. Though both modalities are important for saliency prediction, little work has been done by joint ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Modular segmentation, spatial analysis and visualization of volume electron microscopy datasets

Martin Weigert

Volume electron microscopy is the method of choice for the in situ interrogation of cellular ultrastructure at the nanometer scale, and with the increase in large raw image datasets generated, improving computational strategies for image segmentation and s ...

Nature Portfolio2024

Silicon CMOS and InGaAs(P)/InP SPADs for NIR/SWIR detection

Utku Karaca

Applications demanding imaging at low-light conditions at near-infrared (NIR) and short-wave infrared (SWIR) wavelengths, such as quantum information science, biophotonics, space imaging, and light detection and ranging (LiDAR), have accelerated the develo ...

EPFL2024

Imaging sensor device using an array of single-photon avalanche diode photodetectors

Edoardo Charbon, Andrei Ardelean

The invention relates to an Imaging sensor device in a stacked arrangement comprising: - a pixel array tier comprising a plurality of pixel segments each having a plurality of pixels for photon detection each providing a digital pixel output; - a processin ...

2024

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

Jan Sickmann Hesthaven, Federico Pichi

The present work proposes a framework for nonlinear model order reduction based on a Graph Convolutional Autoencoder (GCA-ROM). In the reduced order modeling (ROM) context, one is interested in obtaining real -time and many-query evaluations of parametric ...

Academic Press Inc Elsevier Science2024

Long plasma duration operation analyses with an international multi-machine (tokamaks and stellarators) database

Stefano Coda, Jeffrey Huang

Combined high-fusion performance and long-pulse operation is one of the key integration challenges for fusion energy development in magnetic devices. Addressing these challenges requires an integrated vision of physics and engineering aspects with the purp ...

IOP Publishing Ltd2024

Generalization and Personalization of Machine Learning for Multimodal Mobile Sensing in Everyday Life

Lakmal Buddika Meegahapola

A range of behavioral and contextual factors, including eating and drinking behavior, mood, social context, and other daily activities, can significantly impact an individual's quality of life and overall well-being. Therefore, inferring everyday life aspe ...

EPFL2024

Development and clinical validation of computational imaging biomarkers for neurodegenerative diseases

Veronica Lily Ravano

Neurodegenerative and neuroinflammatory disorders often involve complex pathophysiological mechanisms that are â to this date â only partially understood. A more comprehensive understanding of those microstructural processes and their characterization ...

EPFL2024

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Volkan Cevher, Grigorios Chrysos, Fanghui Liu

Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate scaled ResNet in the limit of infinitely deep and wide neural networks, of wh ...

2024

Nonlinear simulation of plasma turbulence using a gyrokinetic moment-based approach

Antoine Cyril David Hoffmann

Plasma turbulence plays a fundamental role in determining the performances of magnetic confinement fusion devices, such as tokamaks. Advances in computer science, combined with the development of efficient physical models, have significantly improved our u ...

EPFL2024

Data for Paper "Scalable Semantic 3D Mapping of Coral Reefs with Deep Learning"

Anders Meibom, Devis Tuia, Guilhem Maurice Louis Banc-Prandi, Jonathan Paul Sauder

Example Data for DeepReefMap This dataset contains input videos in MP4 format taken with GoPro Hero 10 Cameras in Reefs in the Red Sea to demonstrate the DeepReefMap tool, which is described in the paper "Scalable Semantic 3D Mapping of Coral Reefs with De ...

EPFL Infoscience2024

Tangentially viewing phase contrast imaging for turbulence measurements in toroidal fusion devices

Stefano Coda

Phase contrast imaging (PCI) is an established and powerful technique for measuring density fluctuations in plasmas and has been successfully applied to several fusion devices. Rooted in a concept first developed for microscopy, PCI belongs to the category ...

2024

Global fluid simulations of plasma turbulence in stellarators

António João Caeiro Heitor Coelho

In order to cope with the decarbonization challenge faced by many countries, fusion is one of the few alternatives to fossil fuels for the production of electricity. Two devices invented in the middle of the previous century have emerged as the most promis ...

EPFL2024

Shape Holomorphy of Boundary Integral Operators on Multiple Open Arcs

Fernando José Henriquez Barraza

We establish shape holomorphy results for general weakly- and hyper-singular boundary integral operators arising from second-order partial differential equations in unbounded two-dimensional domains with multiple finite-length open arcs. After recasting th ...

Springer Birkhauser2024