Publications by Caglar Gulcehre

Related publications (6)

No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO

Andréa Serge Alfredo Miele, Daniil Pyatko, Caglar Gulcehre, Skander Moalla

Reinforcement learning (RL) is inherently rife with non-stationarity since the states and rewards the agent observes during training depend on its changing policy. Therefore, networks in deep RL must be capable of adapting to new observations and fitting n ...

2024

Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers

Xiuying Wei, Caglar Gulcehre, Skander Moalla

State-of-the-art results in large language models (LLMs) often rely on scale, which becomes computationally expensive. This has sparked a research agenda to reduce these models' parameter counts and computational costs without significantly impacting their ...

2024

Self-Recognition in Language Models

Veniamin Veselovskyy, Tim Ruben Davidson, Viacheslav Surkov, Robert West, Caglar Gulcehre, Giuseppe Russo

A rapidly growing number of applications rely on a small set of closed-source language models (LMs). This dependency might introduce novel security risks if LMs develop self-recognition capabilities. Inspired by human identity verification methods, we prop ...

Association for Computational Linguistics2024

Beyond Autoregression: Fast LLMs via Self-Distillation Through Time

Caglar Gulcehre, Justin Samuel Deschenaux

Autoregressive (AR) Large Language Models (LLMs) have demonstrated significant success across numerous tasks. However, the AR modeling paradigm presents certain limitations; for instance, contemporary autoregressive LLMs are trained to generate one token a ...

ICLR2025

SIMPLE HIERARCHICAL PLANNING WITH DIFFUSION

Caglar Gulcehre

Diffusion-based generative methods have proven effective in modeling trajectories with offline datasets. However, they often face computational challenges and can falter in generalization, especially in capturing temporal abstractions for long- horizon tas ...

2024

Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World Models

Caglar Gulcehre

Systematic compositionality, or the ability to adapt to novel situations by creating a mental model of the world using reusable pieces of knowledge, remains a significant challenge in machine learning. While there has been considerable progress in the lang ...

2023