Publications associées à Shape optimisation of technical devices via gradient descent using convolutional neural network proxies

Deep Learning Theory Through the Lens of Diagonal Linear Networks

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Residual-based attention in physics-informed neural networks

Nikolaos Stergiopoulos, Sokratis Anagnostopoulos

Driven by the need for more efficient and seamless integration of physical models and data, physics -informed neural networks (PINNs) have seen a surge of interest in recent years. However, ensuring the reliability of their convergence and accuracy remains ...

Elsevier Science Sa2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Scott William Pesme, Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Topics in statistical physics of high-dimensional machine learning

Hugo Chao Cui

In the past few years, Machine Learning (ML) techniques have ushered in a paradigm shift, allowing the harnessing of ever more abundant sources of data to automate complex tasks. The technical workhorse behind these important breakthroughs arguably lies in ...

EPFL2024

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Volkan Cevher, Grigorios Chrysos, Fanghui Liu

Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate scaled ResNet in the limit of infinitely deep and wide neural networks, of wh ...

2024

Visualization and modelling of dynamic flow in fibrous preforms for liquid composite molding

Helena Luisa Teixido Pedarros

Nowadays, Liquid Composite Molding techniques are often used to manufacture high quality fiber reinforced composite parts at a relatively low cost. These involve an infiltration process, in which a liquid resin is forced to ingress into a dry fibrous porou ...

EPFL2023

Influence of Viscous Lubricant on Nucleation and Propagation of Frictional Ruptures

Marie Estelle Solange Violay, François Xavier Thibault Passelègue, Federica Paglialunga

Fluids are pervasive in the Earth's crust and saturate fractures and faults. The combination of fluids and gouge layers developing along faults can generate fluids of different viscosities. Such viscous fluids were found to influence the reactivation, fric ...

2023

Permeability sets the linear path instability of buoyancy-driven disks

François Gallaire, Pier Giuseppe Ledda, Giuseppe Antonio Zampogna, Giovanni Vagnoli

The prediction of trajectories of buoyancy-driven objects immersed in a viscous fluid is a key problem in fluid dynamics. Simple-shaped objects, such as disks, present a great variety of trajectories, ranging from zig-zag to tumbling and chaotic motions. Y ...

Cambridge University Press2023

Measurement and modelling of dynamic fluid saturation in carbon reinforcements

Véronique Michaud, Baris Çaglar, Helena Luisa Teixido Pedarros, Guillaume Clément Broggi

We propose a methodology to monitor the progressive saturation of a non-translucent unidirectional carbon fabric stack through its thickness by means of X-ray radiography and extract the dynamic saturation curves using image analysis. Four constant flow ra ...

2023

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Giancarlo Ferrari Trecate, Clara Lucía Galimberti, Liang Xu, Luca Furieri

Deep Neural Networks (DNNs) training can be difficult due to vanishing and exploding gradients during weight optimization through backpropagation. To address this problem, we propose a general class of Hamiltonian DNNs (H-DNNs) that stem from the discretiz ...

2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Predicting nonlinear optical scattering with physics-driven neural networks

Demetri Psaltis, Ahmed Ayoub, Carlo Gigli

Deep neural networks trained on physical losses are emerging as promising surrogates for nonlinear numerical solvers. These tools can predict solutions to Maxwell's equations and compute gradients of output fields with respect to the material and geometric ...

AIP Publishing2023

Polynomial-time universality and limitations of deep learning

Emmanuel Abbé

The goal of this paper is to characterize function distributions that general neural networks trained by descent algorithms (GD/SGD), can or cannot learn in polytime. The results are: (1) The paradigm of general neural networks trained by SGD is poly-time ...

WILEY2023

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

EPFL2023

Time Series Analysis of Urban Liveability

Devis Tuia, Diego Marcos Gonzalez, Alex Hubertus Levering

In this paper we explore deep learning models to monitor longitudinal liveability changes in Dutch cities at the neighbourhood level. Our liveability reference data is defined by a country-wise yearly survey based on a set of indicators combined into a liv ...

IEEE2023

Low-Temperature Dynamics of Water Confined in Unidirectional Hydrophilic Zeolite Nanopores

Livia Eleonora Bove Kado, Umbertoluca Ranieri, Maria Rescigno

The dynamical properties of water molecules confinedin the unidirectionalhydrophilic nanopores of AlPO4-54 are investigated withquasi-elastic neutron scattering as a function of temperature downto 118 K. AlPO4-54 has among the largest pores known foralumin ...

AMER CHEMICAL SOC2023

Probabilistic methods for neural combinatorial optimization

Nikolaos Karalias

The monumental progress in the development of machine learning models has led to a plethora of applications with transformative effects in engineering and science. This has also turned the attention of the research community towards the pursuit of construc ...

EPFL2023

Fundamental Limits in Statistical Learning Problems: Block Models and Neural Networks

Elisabetta Cornacchia

This thesis focuses on two selected learning problems: 1) statistical inference on graphs models, and, 2) gradient descent on neural networks, with the common objective of defining and analysing the measures that characterize the fundamental limits. In th ...

EPFL2023

Deep Learning for 3D Surface Modelling and Reconstruction

Benoît Alain René Guillard

In recent years, there has been a significant revolution in the field of deep learning, which has demonstrated its effectiveness in automatically capturing intricate patterns from large datasets. However, the majority of these successes in Computer Vision ...

EPFL2023

Stable Nonconvex-Nonconcave Training via Linear Interpolation

Volkan Cevher, Thomas Michaelsen Pethick, Wanyun Xie

This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss lan ...

2023