Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformers

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

A search is presented for an extended Higgs sector with two new particles, (Formula presented) and (Formula presented), in the process (Formula presented). Novel neural networks classify events with diphotons that are merged and determine the diphoton mass ...

This paper assesses the feasibility of estimating the position and attitude of a target satellite using computer vision, not during proximity operations, but during conjunctions at distances greater than 1 km. The selected target is a small 5 × 5 × 5 cm3 P ...

The ability to reason, plan and solve highly abstract problems is a hallmark of human intelligence. Recent advancements in artificial intelligence, propelled by deep neural networks, have revolutionized disciplines like computer vision and natural language ...

The desire and ability to place AI-enabled applications on the edge has grown significantly in recent years. However, the compute-, area-, and power-constrained nature of edge devices are stressed by the needs of the AI-enabled applications, due to a gener ...

Modern integrated circuits are tiny yet incredibly complex technological artifacts, composed of millions and billions of individual structures working in unison. Managing their complexity and facilitating their design drove part of the co-evolution of mode ...

We present a finite elements-neural network approach for the numerical approximation of parametric partial differential equations. The algorithm generates training data from finite element simulations, and uses a data -driven (supervised) feedforward neura ...

Feedback of power during running is a promising tool for training and determining pacing strategies. However, current power estimation methods show low validity and are not customized for running on different slopes. To address this issue, we developed thr ...

In this thesis we present three closed form approximation methods for portfolio valuation and risk management. The first chapter is titled ``Kernel methods for portfolio valuation and risk management'', and is a joint work with Damir Filipovi'c (SFI and ...

Over the last two decades, data-powered machine learning (ML) tools have profoundly transformed numerous scientific fields. In computational chemistry, machine learning applications have permitted faster predictions of chemical properties and provided powe ...

In the CORTEX project, methods to simulate neutron flux oscillations were enhanced and machine-learning based tools to determine the causes of measured neutron flux oscillations were developed, using the results of simulations as training and validation da ...

Atomic environment fingerprints are widely used in computational materials science, from machine learning potentials to the quantification of similarities between atomic configurations. Many approaches to the construction of such fingerprints, also called ...

Machine Translation (MT) has made considerable progress in the past two decades, particularly after the introduction of neural network models (NMT). During this time, the research community has mostly focused on modeling and evaluating MT systems at the se ...

Machine Learning, or the learning methods used to constitute what is called Artificial Intelligence, is more diversified than is generally presented. Based in particular on the classification proposed by Domingos, the article begins by presenting different ...

Neural networks have become indispensable for a wide range of applications, but they suffer from high computational- and memory-requirements, requiring optimizations from the algorithmic description of the network to the hardware implementation. Moreover, ...

This tutorial covers biomedical image reconstruction, from the foundational concepts of system modeling and direct reconstruction to modern sparsity and learning-based approaches. Imaging is a critical tool in biological research and medicine, and most ima ...

In this paper, we introduce our recent studies on human perception in audio event classification. In particular, the pre-trained model VGGish is used as feature extractor to process audio data, and DenseNet is trained by and used as feature extractor for o ...

The main focus of this research is the development of real-time recording and analysis devices and algorithms for detecting Electroencephalography (EEG) correlates of steering actions. First, offline analysis where performed in a high fidelity driving simu ...

We consider the estimation of multiple time-domain sparse filters from echoic mixtures of several unknown sources, when the sources are sparse in the time-frequency domain. We propose a sparse filter estimation framework consisting of two steps: a) a clust ...

The work described in this thesis takes place in the context of capturing real-life audio for the analysis of spontaneous social interactions. Towards this goal, we wish to capture conversational and ambient sounds using portable audio recorders. Analysis ...

Background: Computational inference of transcriptional regulatory networks remains a challenging problem, in part due to the lack of strong network models. In this paper we present evolutionary approaches to improve the inference of regulatory networks for ...

Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformers

Chat with Graph Search

Search for New Resonances Decaying to Pairs of Merged Diphotons in Proton-Proton Collisions at (Formula presented)

Tiny satellites, big challenges: A feasibility study of machine vision pose estimation for PocketQubes during conjunctions

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

Exploring High-Performance and Energy-Efficient Architectures for Edge AI-Enabled Applications

Graph generative deep learning models with an application to circuit topologies

Error assessment of an adaptive finite elements-neural networks method for an elliptic parametric PDE

Estimation of horizontal running power using foot-worn inertial measurement units

Closed form approximation methods for portfolio valuation and risk management

Kernel Methods and Similarity Learning Applied in Computational Chemistry

Sensitivity analysis in core diagnostics

An assessment of the structural resolution of various fingerprints commonly used in machine learning

Discourse Phenomena in Machine Translation

How Machine Learning is challenged by algorithmic complexity classes. From models to practices

Lupulus: A Flexible Hardware Accelerator For Neural Networks

Biomedical Image Reconstruction: From the Foundations to Deep Neural Networks

Deep Learning of Human Perception in Audio Event Classification

Detecting EEG correlates during preparation of complex driving maneuvers

Doubly sparse models for multiple filter estimation in sparse echoic environments

Privacy-Sensitive Audio Features for Conversational Speech Processing

Refining transcriptional regulatory networks using network evolutionary models and gene histories