Publications associées à Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

Publications associées (18)

Machine learning enhanced atom probe tomography analysis

Atom probe tomography (APT) is a burgeoning characterization technique that provides compositional mapping of materials in three-dimensions at near-atomic scale. Since its significant expansion in the past 30 years, we estimate that one million APT dataset ...

2026

Hadamard product in deep learning: Introduction, Advances and Challenges

Volkan Cevher, Yongtao Wu

While convolution and self-attention mechanisms have dominated architectural design in deep learning, this survey examines a fundamental yet understudied primitive: the Hadamard product. Despite its widespread implementation across various applications, th ...

2025

Neural networks trained with SGD learn distributions of increasing complexity

Maria Refinetti

The uncanny ability of over-parameterised neural networks to generalise well has been explained using various ‘simplicity biases’. These theories postulate that neural networks avoid overfitting by first fitting simple, linear classifiers before learning m ...

2025

Robustly Learning Regions of Attraction From Fixed Data

Colin Neil Jones

While stability analysis is a mainstay for control science, especially computing regions of attraction of equilibrium points, until recently most stability analysis tools always required explicit knowledge of the model or a high-fidelity simulator represen ...

2025

Subspace embedding with random Khatri–Rao products and its application to eigensolvers

Daniel Kressner, Hei Yin Lam

Various iterative eigenvalue solvers have been developed to compute parts of the spectrum for a large sparse matrix, including the power method, Krylov subspace methods, contour integral methods and preconditioned solvers such as the so-called locally opti ...

Oxford University Press (OUP)2025

Enhancing Lift and Dynamic Stability Through Morphological Optimization of Wings for Tailless Flapping-Wing Robots

Josephine Anna Eleanor Hughes

The design of wings is crucial for tailless Flapping-wing robots (FWRs), as these robots rely exclusively on their wings for lift generation and body control. However, the current methodology for wing design primarily depends on the experience and heuristi ...

2025

Semantic Segmentation of Benthic Classes in Reef Environments Using a Large Vision Transformer

Devis Tuia, Jonathan Paul Sauder, Charlotte Louise Sertic

Coral reefs are crucial for biodiversity and provide vital resources for humankind. But despite such a central role, they are confronted to increasing threats linked to climate change, pollution, and local stressors. To ensure effective conservation, effic ...

Springer Science and Business Media Deutschland GmbH2025

Personalizing Learning and Empowering Educators through Multi-Dimensional Clustering, Teacher Dashboards, and AI-Hybrid Learning Systems

Paola Mejia Domenzain

The shift from traditional classroom settings to technology-supported learning environments has led to the adoption of learning analytics and artificial intelligence (AI) in education. These technologies promise to support personalized learning by analyzin ...

EPFL2025

Representing and machine learning complex electronic-structure properties of materials

Austin Jacob Zadoks

The rise of high-throughput first-principles calculations of materials since the turn of the millennium has gifted the fields of physics, quantum chemistry, and materials science with a mountainous and ever-growing pile of data ready to be mined for hidden ...

EPFL2025

The Galaxy Activity, Torus, and Outflow Survey (GATOS) VI. Black hole mass estimation using machine learning

Rémi Henri Philippe Poitevineau

The detailed feeding and feedback mechanisms of active galactic nuclei (AGNs) are not yet well known. For low-luminosity AGNs, obscured AGNs, and late-type galaxies, the masses of their central black holes (BH) are difficult to determine precisely. Our goa ...

EDP SCIENCES S A2025

Higher-order knowledge-enhanced recommendation with heterogeneous hypergraph multi-attention

Thanh Trung Huynh

Recent advancements in recommender systems have focused on integrating knowledge graphs (KGs) to leverage their auxiliary information. The core idea of KG-enhanced recommenders is to incorporate rich semantic information for more accurate recommendations. ...

2024

Monitoring of water volume in a porous reservoir using seismic data: A 3D simulation study

Jan Sickmann Hesthaven

A potential framework to estimate the volume of water stored in a porous storage reservoir from seismic data is neural networks. In this study, the man-made groundwater reservoir is modeled as a coupled poroviscoelastic–viscoelastic medium, and the underly ...

2024

LiDAR-NeRF: Novel LiDAR View Synthesis via Neural Radiance Fields

Mathieu Salzmann

We introduce a new task, novel view synthesis for LiDAR sensors. While traditional model-based LiDAR simulators with style-transfer neural networks can be applied to render novel views, they fall short of producing accurate and realistic LiDAR patterns bec ...

Association for Computing Machinery, Inc2024

Unsupervised quality monitoring of metal additive manufacturing using Bayesian adaptive resonance

Patrik Willi Hoffmann

Metal additive manufacturing is a recent breakthrough technology that promises automated production of complex geometric shapes at low operating costs. However, its potential is not yet fully exploited due to the low reproducibility of quality in mass prod ...

CELL PRESS2024

Function Learning with Financial Applications

Paolo Colusso

This thesis sets itself the goal of investigating function learning approaches. The topic is extremely relevant today, since the abundance of data, jointly with the technological progress which has enhanced computing power, now allow to represent functions ...

EPFL2024

Positive-Unlabeled Constraint Learning for Inferring Nonlinear Continuous Constraints Functions From Expert Demonstrations

Baiyu Peng, Aude Billard

Planning for diverse real-world robotic tasks necessitates to know and write all constraints. However, instances exist where these constraints are either unknown or challenging to specify accurately. A possible solution is to infer the unknown constraints ...

2024

Enhancing Explainability and Performance in AI-Driven Face Recognition and Deepfake Detection

Yuhang Lu

The rapid advancement of artificial intelligence (AI) has greatly influenced numerous research areas, leading to significant breakthroughs in human face-related technologies, particularly in face recognition and deepfake detection. While offering substanti ...

EPFL2024

Assessment of fine-tuned large language models for real-world chemistry and material science applications

Berend Smit, Anne-Clémence Corminboeuf, Rubén Laplaza Solanas, Kevin Maik Jablonka, Sauradeep Majumdar, Joren Jozef M. Van Herck, Beatriz Bueno Mouriño, Seyed Amirmohammad Elahi

The current generation of large language models (LLMs) has limited chemical knowledge. Recently, it has been shown that these LLMs can learn and predict chemical properties through fine-tuning. Using natural language to train machine learning models opens ...

ROYAL SOC CHEMISTRY2024