A Theory of Finite-Width Neural Networks: Generalization, Scaling Laws, and the Loss Landscape

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Photometric stereo, a computer vision technique for estimating the 3D shape of objects through images captured under varying illumination conditions, has been a topic of research for nearly four decades. In its general formulation, photometric stereo is an ...

Training data mixtures greatly impact the generalization performance of large language models. Existing domain reweighting methods often rely on costly weight computations and require retraining when new data is introduced. To this end, we introduce a flex ...

Recent advancements in deep learning have revolutionized 3D computer vision, enabling the extraction of intricate 3D information from 2D images and video sequences. This thesis explores the application of deep learning in three crucial challenges of 3D com ...

Implicit neural representations map a shape-specific latent code and a 3D coordinate to its corresponding signed distance (SDF) value. However, this approach only offers a single level of detail. Emulating low levels of detail can be achieved with shallow ...

In this thesis we explore the applications of projective geometry, a mathematical theory of the relation between 3D scenes and their 2D images, in modern learning-based computer vision systems. This is an interesting research question which contradicts the ...

Understanding the structure of real data is paramount in advancing modern deep-learning methodologies. Natural data such as images are believed to be composed of features organized in a hierarchical and combinatorial manner, which neural networks capture d ...

Deriving health indicators of rotating machines is crucial for their maintenance. However, this process is challenging for the prevalent adopted intelligent methods since they may take the whole data distributions, not only introducing noise interference b ...

We study the generalization of iterative noisy gradient schemes on smooth non-convex losses. Formally, we establish time-independent information theoretic generalization bounds for Stochastic Gradient Langevin Dynamics (SGLD) that do not diverge as the ite ...

Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence of benchmark evaluations and theoretical guarantees for searching these robus ...

The process of protein aggregation, central to neurodegenerative diseases like Huntington’s, is challenging to study due to its unpredictable nature and relatively rapid kinetics. Understanding its biomechanics is crucial for unraveling its role in disease ...

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

As a data-driven approach, multi-agent reinforcement learning (MARL) has made remarkable advances in solving cooperative residential load scheduling problems. However, centralized training, the most common paradigm for MARL, limits large-scale deployment i ...

Machine learning (ML) enables artificial intelligent (AI) agents to learn autonomously from data obtained from their environment to perform tasks. Modern ML systems have proven to be extremely effective, reaching or even exceeding human intelligence. Altho ...

Dynamic aperture is an important concept for the study of non-linear beam dynamics in circular accelerators. It describes the extent of the phase-space region where a particle's motion remains bounded over a given number of turns. Understanding the feature ...

Accurate segmentation of anatomical substructures within 3D curvilinear structures in medical imaging remains challenging due to their complex geometry and the scarcity of diverse, large-scale datasets for algorithm development and evaluation. In this pape ...

The desire and ability to place AI-enabled applications on the edge has grown significantly in recent years. However, the compute-, area-, and power-constrained nature of edge devices are stressed by the needs of the AI-enabled applications, due to a gener ...

Visual affordance segmentation identifies image regions of an object an agent can interact with. Existing methods re-use and adapt learning-based architectures for semantic segmentation to the affordance segmentation task and evaluate on small-size dataset ...

A Theory of Finite-Width Neural Networks: Generalization, Scaling Laws, and the Loss Landscape

Chat with Graph Search

Aggregating Spatial and Photometric Context for Photometric Stereo

CHAMELEON: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning

Advancing Self-Supervised Deep Learning for 3D Scene Understanding

A Latent Implicit 3D Shape Model for Multiple Levels of Detail

Incorporating Projective Geometry into Deep Learning

A phase transition in diffusion models reveals the hierarchical nature of data

Classifier-free diffusion-based weakly-supervised approach for health indicator derivation in rotating machines: Advancing early fault detection and condition monitoring

Generalization of Noisy SGD in Unbounded Non-convex Settings

Robust NAS under adversarial training: benchmark, theory, and beyond

Self-driving microscopy detects the onset of protein aggregation and enables intelligent Brillouin imaging

Efficient local linearity regularization to overcome catastrophic overfitting

Scalable Multi-agent Reinforcement Learning for Residential Load Scheduling Under Data Governance

Few-shot Learning for Efficient and Effective Machine Learning Model Adaptation

Optimizing Dynamic Aperture Studies with Active Learning

Frenet-Serret Frame-based Decomposition for Part Segmentation of 3D Curvilinear Structures

Exploring High-Performance and Energy-Efficient Architectures for Edge AI-Enabled Applications

Segmenting Object Affordances: Reproducibility and Sensitivity to Scale