On the Convergence of Encoder-only Shallow Transformers

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

The application of machine learning in medicine and healthcare has led to the creation of numerous diagnostic and prognostic models. However, despite their success, current approaches generally issue predictions using data from a single modality. This stan ...

Deep neural networks often fail to generalize when deployed in dynamic and unpredictable real-world environments, where data distributions at test time can differ significantly from those encountered during training. Since simulating all possible distribut ...

DARTS-NETGAB is a unified framework for real-time simulation and automated design of gas-bearing supported turbocompressors, facilitating efficient transition from optimization to manufacturable designs. The framework integrates ensemble artificial neural ...

Previous methods for predicting room acoustic parameters and speech quality metrics have focused on the single-channel case, where room acoustics and Mean Opinion Score (MOS) are predicted for a single recording device. However, quality-based device select ...

Deep learning has achieved remarkable success in diverse fields in recent years. However, this growth presents significant challenges, particularly in terms of energy consumption during both training and inference phases. While there have been efforts to i ...

This study introduces a Machine Learning (ML) approach for estimating the T2 spectrum and myelin water fraction (MWF) using multi-echo T2 (MET2) data from preclinical 7T Magnetic Resonance Imaging (MRI) scanners. ML methods have shown promise in MWF estima ...

The contextual stochastic block model (CSBM) was proposed for unsupervised community detection on attributed graphs where both the graph and the high-dimensional node information correlate with node labels. In the context of machine learning on graphs, the ...

The increasing availability of Massive Open Online Courses (MOOCs) has created a necessity for personalized course recommendation systems. These systems often combine neural networks with Knowledge Graphs (KGs) to achieve richer representations of learners ...

The growing availability of distributed data has led to the increased use of machine learning (ML) algorithms in distributed topologies, where multiple nodes collaborate to train models under the coordination of a central server. However, distributed learn ...

Multiple drug concentrations concurrent detection and quantification based on electrochemical sensors are of great importance for therapeutic drug monitoring (TDM) and the development of personalized therapy. Cyclic voltammogram (CV) results obtained by el ...

Novice students in law courses or students who encounter legal education face the challenge of acquiring specialized and highly concept-oriented knowledge. Structured and persuasive writing combined with the necessary domain knowledge is challenging for ma ...

Sharpness-aware minimization (SAM) has been shown to improve the generalization of neural networks. However, each SAM update requires sequentially computing two gradients, effectively doubling the per-iteration cost compared to base optimizers like SGD. We ...

Motivation is a multifaceted phenomenon that we explore within the framework of decision-making. Through this cognitive process, actions are directed towards specific goals by performing a trade-off between the cost and benefit of an action. The dorsomedia ...

Dynamic human energy expenditure (EE) consists of energy costs for resting metabolism, food digestion, physical activity, and thermoregulation. Currently, multiple models predict EE mainly concerning physical activity, thus, discarding other factors contri ...

Identifying manipulated regions in images is a challenging task due to the existence of very accurate image inpainting techniques leaving almost unnoticeable traces in tampered regions. These image inpainting methods can be used for multiple purposes (e.g. ...

The European Space Agency's Euclid mission will provide high-quality imaging for about 1.5 billion galaxies. A software pipeline to automatically process and analyse such a huge amount of data in real time is being developed by the Science Ground Segment o ...

Coded aperture cameras can be manufactured cheaply, have a very thin form-factor, and may be transparent and flexible, thus providing easy-to-integrate and compact image sensors. However, limitations in reconstructed image quality and resolution have imped ...

Under seismic actions, stone masonry buildings are prone to damage. To assess the severity of damaged masonry buildings and their failure modes, engineers connect these problems to surface crack features, such as the crack width and the extent of cracking. ...

Supervised learning is constrained by the availability of labeled data, which are especially expensive to acquire in the field of digital pathology. Making use of open-source data for pre-training or using domain adaptation can be a way to overcome this is ...

Deep convolutional neural networks (DCNNs) are currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. The success of DCNNs can be attributed to the careful selection of their buil ...

On the Convergence of Encoder-only Shallow Transformers

Chattez avec Graph Search

Automated Ensemble Multimodal Machine Learning for Healthcare

Adaptive Test-Time Learning Under Domain Shift: From Augmentation to Prolonged Adaptation

DARTS-NETGAB: Design Automation and Real-Time Simulation Using Neural Network Ensembles for Turbocompressors on Gas-Bearings

Multi-Channel Mosra: Mean Opinion Score and Room Acoustics Estimation Using Simulated Data and A Teacher Model

Backpropagation-free Training of Analog AI Accelerators

In-Silico Trained AI for Enhanced T2Spectrum Imaging and Myelin Water Fraction Mapping in Preclinical 7T MRI

Optimal Inference in Contextual Stochastic Block Models

Finding Paths for Explainable MOOC Recommendation: A Learner Perspective

Towards Practical Homomorphic Aggregation in Byzantine-Resilient Distributed Learning

Automatic simulation of electrochemical sensors by machine learning for drugs quantification

LegalWriter: An Intelligent Writing Support System for Structured and Persuasive Legal Case Writing for Novice Law Students

SAMPa: Sharpness-aware Minimization Parallelized

Neurobiological and metabolic basis of motivation and effort-based decision-making - A human 1H-MRS, fMRI and machine learning study

Dynamic personalized human body energy expenditure: Prediction using time series forecasting LSTM models

IRL-Net: Inpainted Region Localization Network via Spatial Attention

Euclid preparation XXV. The Euclid Morphology Challenge: Towards model-fitting photometry for billions of galaxies

Toward High-Resolution Face Image Generation From Coded Aperture Camera

Machine-learning for damage assessment of rubble stone masonry piers based on crack patterns

Self-rule to multi-adapt: Generalized multi-source feature learning using unsupervised domain adaptation for colorectal cancer tissue detection

Deep Polynomial Neural Networks