Course

EE-628: Training Large Language Models

Summary

This PhD-level course dives deep into the training of Large Language Models (LLMs), focusing on the complementary roles of datasets, pre-training and post training methodologies in shaping model performance and scalability.

Lectures

Instructors (2)

Volkan Cevher

Volkan Cevher received the B.Sc. (valedictorian) in electrical engineering from Bilkent University in Ankara, Turkey, in 1999 and the Ph.D. in electrical and computer engineering from the Georgia Institute of Technology in Atlanta, GA in 2005. He was a Research Scientist with the University of Maryland, College Park from 2006-2007 and also with Rice University in Houston, TX, from 2008-2009. Currently, he is an Associate Professor at the Swiss Federal Institute of Technology Lausanne and a Faculty Fellow in the Electrical and Computer Engineering Department at Rice University. His research interests include machine learning, signal processing theory, optimization theory and methods, and information theory. Dr. Cevher is an ELLIS fellow and was the recipient of the Google Faculty Research award in 2018, the IEEE Signal Processing Society Best Paper Award in 2016, a Best Paper Award at CAMSAP in 2015, a Best Paper Award at SPARS in 2009, and an ERC CG in 2016 as well as an ERC StG in 2011.

Caglar Gulcehre

Teaching & PhD PhD Students Tim Ruben Davidson, Skander Moalla, Anja Surina, Xiuying Wei, Liangze Jiang, Mikhail Terekhov, Amin Mansouri, Justin Samuel Deschenaux, Yitao Xu Courses Training Large Language Models EE-628 This PhD-level course dives deep into the training of Large Language Models (LLMs), focusing on the complementary roles of datasets, pre-training and post training methodologies in shaping model performance and scalability.

Official source

Moodle Page

http://go.epfl.ch/EE-628

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Summary

Lectures

Instructors (2)

Volkan Cevher

Caglar Gulcehre

Official source

Moodle Page

http://go.epfl.ch/EE-628

Related courses (31)

EE-556: Mathematics of data: from theory to computation

This course provides an overview of key advances in continuous optimization and statistical analysis for machine learning. We review recent learning formulations and models as well as their guarantees

PHYS-467: Machine learning for physicists

Machine learning and data analysis are becoming increasingly central in sciences including physics. In this course, fundamental principles and methods of machine learning will be introduced and practised. ...

CS-401: Applied data analysis

This course teaches the basic techniques, methodologies, and practical skills required to draw meaningful insights from a variety of data, with the help of the most acclaimed software tools in the data science world (pandas, scikit-learn, Spark, etc.) ...

CIVIL-226: Introduction to machine learning for engineers

Machine learning is a sub-field of Artificial Intelligence that allows computers to learn from data, identify patterns and make predictions. As a fundamental building block of the Computational Thinking education at EPFL, Civil students will learn ML with ...

CS-423: Distributed information systems

This course introduces the foundations of information retrieval, data mining and knowledge bases, which constitute the foundations of today's Web-based distributed information systems. ...