On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Contemporary artificial intelligence (AI) models' exponentially growing complexity translates to their expanding skill sets. They can create language or image content mimicking human creativity and understanding, becoming indispensable tools in education, ...

Having a single neural network to handle a wide and varied range of tasks and modalities has been a long-standing goal. Such a model brings notable advantages, such as test-time computational efficiency, modality fusion, and reduced model size. Our goal in ...

Segmenting brain tumors is a critical task in medical imaging that relies on advanced deep-learning methods. However, effectively handling complex tumor regions requires more comprehensive and advanced strategies to overcome challenges such as computationa ...

In this concise contribution, it is demonstrated that Recurrent Neural Networks (RNNs) based on Gated Recurrent Unit (GRU) architecture, possess the capability to learn the complex dynamics of rate-and-state friction (RSF) laws from synthetic data. The dat ...

Computing has long been central to technological advancements, where each leap in compute power has catalyzed applications once thought impossible, such as smartphones. Recently, the exponential growth in the compute demand of artificial intelligence (AI) ...

Machine translation (MT) systems and language models (LMs) have demonstrated impressive performance in recent years, driven by advances in architecture, algorithms, and the scaling of parameters and training data. Despite the ability of these models to int ...

Finding proper collective variables for complex systems and processes is one of the most challenging tasks in simulations, which limits the interpretation of experimental and simulated data and the application of enhanced sampling techniques. Here, we prop ...

In this study, we employed traditional methods and deep learning models to improve resolution and quality of low-resolution AFM images made under standard ambient scanning. Both traditional methods and deep learning models were benchmarked and quantified r ...

The increasing integration of Distributed Energy Resources (DERs) within power distribution grids introduces uncertainties that can result in control and operation issues such as line congestions, reduced voltage quality and increased grid imbalance cost. ...

Research in speech processing has largely focused on source-system modeling, where vocal fold vibrations serve as the source and the vocal cavity's articulations as the system. Nonetheless, speech production includes a complex combination of physiological ...

This letter investigates the universal approximation capabilities of Hamiltonian Deep Neural Networks (HDNNs) that arise from the discretization of Hamiltonian Neural Ordinary Differential Equations. Recently, it has been shown that HDNNs enjoy, by design, ...

The great potential brought by large-scale data in the humanities is still hindered by the time and technicality required for making documents digitally intelligible. Within urban studies, historical cadasters have been hitherto largely under-explored desp ...

De novo protein design enhances our understanding of the principles that govern protein folding and interactions, and has the potential to revolutionize biotechnology through the engineering of novel protein functionalities. Despite recent progress in comp ...

This paper presents an acoustic impedance control architecture for an electroacoustic absorber combining both feedforward and feedback microphone-based strategies on a current-driven loudspeaker. Feedforward systems enable good performance for direct imped ...

On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks

Chat with Graph Search

Programming Light Propagation for Efficient Artificial Intelligence

Scaling the Modalities in Multimodal Foundation Models

LIU-NET: lightweight Inception U-Net for efficient brain tumor segmentation from multimodal 3D MRI images

Data-driven dynamic friction models based on Recurrent Neural Networks

Optical Processors for AI Computing and Networking

From Preprocessing to Evaluation: The Role of External Knowledge in Advancing Language Models and Machine Translation

Inherent Structural Descriptors Via Machine Learning

Deep learning for enhancement of low-resolution and noisy scanning probe microscopy images

Local market-aware optimal allocation of energy storage systems considering price fairness in power distribution networks

Towards Integrated Processing of Physiological Signals and Speech

Universal Approximation Property of Hamiltonian Deep Neural Networks

Effective annotation for the automatic vectorization of cadastral maps

De novo protein design by inversion of the AlphaFold structure prediction network

Robust direct acoustic impedance control using two microphones for mixed feedforward-feedback controller