Concept

Loss function

Related publications (31)

Extensions of Peer Prediction Incentive Mechanisms

As large, data-driven artificial intelligence models become ubiquitous, guaranteeing high data quality is imperative for constructing models. Crowdsourcing, community sensing, and data filtering have long been the standard approaches to guaranteeing or imp ...

EPFL2024

Stability: a search for structure

Wouter Jongeneel

In this thesis we study stability from several viewpoints. After covering the practical importance, the rich history and the ever-growing list of manifestations of stability, we study the following. (i) (Statistical identification of stable dynamical syste ...

EPFL2024

Efficient local linearity regularization to overcome catastrophic overfitting

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

2024

Random matrix methods for high-dimensional machine learning models

Antoine Philippe Michel Bodin

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

EPFL2024

Optimal regimes for algorithm-assisted human decision-making

Mats Julius Stensrud, Aaron Leor Sarvet, Julien David Laurendeau

We consider optimal regimes for algorithm-assisted human decision-making. Such regimes are decision functions of measured pre-treatment variables and, by leveraging natural treatment values, enjoy a superoptimality property whereby they are guaranteed to o ...

2024

Less Conservatism, Stronger Robustness: Iterative Robust Gain-Scheduled Path Following Control of Autonomous Bus With Unstructured and Changing Dynamics

Denis Gillet

Path-following control is a critical technology for autonomous vehicles. However, time-varying parameters, parametric uncertainties, external disturbances, and complicated environments significantly challenge autonomous driving. We propose an iterative rob ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Fair real-time control of energy storage systems in active distribution networks in the presence of uncertainties

Plouton Grammatikos

The thesis explores the issue of fairness in the real-time (RT) control of battery energy storage systems (BESSs) hosted in active distribution networks (ADNs) in the presence of uncertainties by proposing and experimentally validating appropriate control ...

EPFL2024

Data-driven modeling of beam loss in the LHC

Tatiana Pieloni, Michael Schenk, Ekaterina Krymova, Loïc Thomas Davies Coyle

In the Large Hadron Collider, the beam losses are continuously measured for machine protection. By design, most of the particle losses occur in the collimation system, where the particles with high oscilla-tion amplitudes or large momentum error are scrape ...

FRONTIERS MEDIA SA2023

Reducibility bounds of objective functions over the integers

Friedrich Eisenbrand, Kim-Manuel Klein

We study the settings where we are given a separable objective function of n variables defined in a given box of integers. We show that in many cases we can replace the given objective function by a new function with a much smaller domain. Our results appl ...

Elsevier2023

Linear-Covariance Loss for End-to-End Learning of 6D Pose Estimation

Mathieu Salzmann, Yinlin Hu

Most modern image-based 6D object pose estimation methods learn to predict 2D-3D correspondences, from which the pose can be obtained using a PnP solver. Because of the non-differentiable nature of common PnP solvers, these methods are supervised via the i ...

Ieee Computer Soc2023

Learning curves for the multi-class teacher-student perceptron

Lenka Zdeborová, Bruno Loureiro, Elisabetta Cornacchia, Rodrigo Soares Veiga

One of the most classical results in high-dimensional learning theory provides a closed-form expression for the generalisation error of binary classification with a single-layer teacher-student perceptron on i.i.d. Gaussian inputs. Both Bayes-optimal (BO) ...

IOP Publishing Ltd2023

Multi-agent Learning with Privacy Guarantees

Elsa Rizk

A multi-agent system consists of a collection of decision-making or learning agents subjected to streaming observations from some real-world phenomenon. The goal of the system is to solve some global learning or optimization problem in a distributed or dec ...

EPFL2023

Topologically Better Delineation of Curvilinear Structures

Doruk Oner

Curvilinear structures are frequently observed in a variety of domains and are essential for comprehending neural circuits, detecting fractures in materials, and determining road and irrigation canal networks. It can be costly and time-consuming to manuall ...

EPFL2023

On the symmetries in the dynamics of wide two-layer neural networks

Lénaïc Chizat

We consider the idealized setting of gradient flow on the population risk for infinitely wide two-layer ReLU neural networks (without bias), and study the effect of symmetries on the learned parameters and predictors. We first describe a general class of s ...

AMER INST MATHEMATICAL SCIENCES-AIMS2023

Bayes-optimal Learning of Deep Random Networks of Extensive-width

Lenka Zdeborová, Florent Gérard Krzakala, Hugo Chao Cui

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width ...

2023

Density estimation in RKHS with application to Korobov spaces in high dimensions

Fabio Nobile, Yoshihito Kazashi

A kernel method for estimating a probability density function from an independent and identically distributed sample drawn from such density is presented. Our estimator is a linear combination of kernel functions, the coefficients of which are determined b ...

2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Distributionally Robust Linear Quadratic Control

Daniel Kuhn, Bahar Taskesen, Cagil Kocyigit

Linear-Quadratic-Gaussian (LQG) control is a fundamental control paradigm that is studied in various fields such as engineering, computer science, economics, and neuroscience. It involves controlling a system with linear dynamics and imperfect observations ...

2023

An Unbiased Method of Measuring the Ratio of Two Data Sets

Jean-Paul Richard Kneib, Huanyuan Shan

In certain cases of astronomical data analysis, the meaningful physical quantity to extract is the ratio R between two data sets. Examples include the lensing ratio, the interloper rate in spectroscopic redshift samples, and the decay rate of gravitational ...

IOP Publishing Ltd2023

Get More for Less in Decentralized Learning Systems

Anne-Marie Kermarrec, Rafael Pereira Pires, Rishi Sharma, Akash Balasaheb Dhasade, Milos Vujasinovic

Decentralized learning (DL) systems have been gaining popularity because they avoid raw data sharing by communicating only model parameters, hence preserving data confidentiality. However, the large size of deep neural networks poses a significant challeng ...

IEEE2023