Publications by Yongtao Wu | EPFL Graph Search

Related publications (8)

Hadamard product in deep learning: Introduction, Advances and Challenges

While convolution and self-attention mechanisms have dominated architectural design in deep learning, this survey examines a fundamental yet understudied primitive: the Hadamard product. Despite its widespread implementation across various applications, th ...

2025

Machine learning-aided hysteretic response prediction of double skin composite wall under earthquake loads

Yongtao Wu

Accurately and rapidly simulating the hysteretic behavior of double skin composite wall (DSCW) under earthquake loads enhance the efficiency of seismic performance assessments for high-rise steel-concrete composite structures. The application of artificial ...

2025

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu, Yongtao Wu

Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds. The study on NTK has been devoted to typical neural network architectures, but it is incomplete for neural networks with Hadamar ...

2022

On the Convergence of Encoder-only Shallow Transformers

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu

In this paper, we aim to build the global convergence theory of encoder-only shallow Transformers under a realistic setting from the perspective of architectures, initialization, and scaling under a finite width regime. The difficulty lies in how to tackle ...

2023

Revisiting Character-level Adversarial Attacks for Language Models

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu, Elias Abad Rocamora

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based methods, are susceptible to altering sentence semantics, leading to invalid adv ...

2024

Single-pass Detection of Jailbreaking Input in Large Language Models

Volkan Cevher, Grigorios Chrysos, Yongtao Wu, Elias Abad Rocamora

Defending aligned Large Language Models (LLMs) against jailbreaking attacks is a challenging problem, with existing approaches requiring multiple requests or even queries to auxiliary LLMs, making them computationally heavy. Instead, we focus on detecting ...

2025

Robust NAS under adversarial training: benchmark, theory, and beyond

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu

Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence of benchmark evaluations and theoretical guarantees for searching these robus ...

2024

Universal Gradient Methods for Stochastic Convex Optimization

Volkan Cevher, Ali Kavis, Kimon Antonakopoulos, Yongtao Wu

We develop universal gradient methods for Stochastic Convex Optimization (SCO). Our algorithms automatically adapt not only to the oracle's noise but also to the Hölder smoothness of the objective function without a priori knowledge of the particular setti ...

2024