Caglar GulcehreTeaching & PhD PhD Students Tim Ruben Davidson, Skander Moalla, Anja Surina, Xiuying Wei, Liangze Jiang, Mikhail Terekhov, Amin Mansouri, Justin Samuel Deschenaux, Yitao Xu Courses Training Large Language Models EE-628 This PhD-level course dives deep into the training of Large Language Models (LLMs), focusing on the complementary roles of datasets, pre-training and post training methodologies in shaping model performance and scalability.
Hervé BourlardTeaching & PhD Past EPFL PhD Students Astrid Hagen, Katrin Weber, Todd Stephenson, Jitendra Ajmera, Shajith Ikbal, Mathew Magimai Doss, Mohamed Faouzi Benzeghiba, Mark Barnard, Yann Rodriguez, Hemant Misra, Christos Dimitrakakis, Guillaume Lathoud, Alexei Pozdnoukhov, Agnès Just, Mikaela Keller, Norman Hoon Thian Poh, Dong Zhang, Florent Monay Michaud, Pedro Manuel Da Silva Quelhas, Kevin Smith, Silèye Oumar Ba, Bertrand Mesot, Hamed Ketabdar, David Grangier, Guillermo Aradilla, Jean-François Paiement, Nico
Martin SchrimpfExpertise
NeuroAI, computational neuroscience, computer vision, natural language processing, human alignment, startups [CV]
Mission
Modeling the brain: Martin's research focuses on a computational understanding of the neural mechanisms underlying natural intelligence in vision and language. To achieve this goal, he bridges Deep Learning, Neuroscience, and Cognitive Science, building artificial neural network models that match the brain’s neural representations in their internal processing and are aligned to
Andrea CavallaroExpertise
I focus on machine learning for audio-visual sensing to enable systems to serve agreed purposes and improve their abilities through interactions with the environment, with other systems and with people. The goal of my research is to create the next-generation machine perception models for the effective use of sensory data, and the safe operation of autonomous systems that gain information about their environment and make decisions in partnership with humans or independently. These models are trans
Jean-Marc OdobezExpertise
Computer Vision, machine Learning (Bayesian models, deep learning), multimodal processing.Activity analysis, human behavior understanding, human communication, interaction modeling
Dr Jean-Marc Odobez (Msc 1990, PhD 1994) received his Ph.D degree from Rennes University in 1994 for his PhD dissertation done at INRIA. After 5 years as an assistant professor at the University du Maine, France, he decided to join Idiap, where he is now the Head of the Perception & Activity Understanding Group and adju
Wulfram GerstnerWulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.
Henry MarkramHenry Markram started a dual scientific and medical career at the University of Cape Town, in South Africa. His scientific work in the 80s revealed the polymodal receptive fields of pontomedullary reticular formation neurons in vivo and how acetylcholine re-organized these sensory maps.
He moved to Israel in 1988 and obtained his PhD at the Weizmann Institute where he discovered a link between acetylcholine and memory mechanisms by being the first to show that acetylcholine modulates the NMDA receptor in vitro studies, and thereby gates which synapses can undergo synaptic plasticity. He was also the first to characterize the electrical and anatomical properties of the cholinergic neurons in the medial septum diagonal band.
He carried out a first postdoctoral study as a Fulbright Scholar at the NIH, on the biophysics of ion channels on synaptic vesicles using sub-fractionation methods to isolate synaptic vesicles and patch-clamp recordings to characterize the ion channels. He carried out a second postdoctoral study at the Max Planck Institute, as a Minerva Fellow, where he discovered that individual action potentials propagating back into dendrites also cause pulsed influx of Ca2 into the dendrites and found that sub-threshold activity could also activated a low threshold Ca2 channel. He developed a model to show how different types of electrical activities can divert Ca2 to activate different intracellular targets depending on the speed of Ca2 influx an insight that helps explain how Ca2 acts as a universal second messenger. His most well known discovery is that of the millisecond watershed to judge the relevance of communication between neurons marked by the back-propagating action potential. This phenomenon is now called Spike Timing Dependent Plasticity (STDP), which many laboratories around the world have subsequently found in multiple brain regions and many theoreticians have incorporated as a learning rule. At the Max-Planck he also started exploring the micro-anatomical and physiological principles of the different neurons of the neocortex and of the mono-synaptic connections that they form - the first step towards a systematic reverse engineering of the neocortical microcircuitry to derive the blue prints of the cortical column in a manner that would allow computer model reconstruction.
He received a tenure track position at the Weizmann Institute where he continued the reverse engineering studies and also discovered a number of core principles of the structural and functional organization such as differential signaling onto different neurons, models of dynamic synapses with Misha Tsodyks, the computational functions of dynamic synapses, and how GABAergic neurons map onto interneurons and pyramidal neurons. A major contribution during this period was his discovery of Redistribution of Synaptic Efficacy (RSE), where he showed that co-activation of neurons does not only alter synaptic strength, but also the dynamics of transmission. At the Weizmann, he also found the tabula rasa principle which governs the random structural connectivity between pyramidal neurons and a non-random functional connectivity due to target selection. Markram also developed a novel computation framework with Wolfgang Maass to account for the impact of multiple time constants in neurons and synapses on information processing called liquid computing or high entropy computing.
In 2002, he was appointed Full professor at the EPFL where he founded and directed the Brain Mind Institute. During this time Markram continued his reverse engineering approaches and developed a series of new technologies to allow large-scale multi-neuron patch-clamp studies. Markrams lab discovered a novel microcircuit plasticity phenomenon where connections are formed and eliminated in a Darwinian manner as apposed to where synapses are strengthening or weakened as found for LTP. This was the first demonstration that neural circuits are constantly being re-wired and excitation can boost the rate of re-wiring.
At the EPFL he also completed the much of the reverse engineering studies on the neocortical microcircuitry, revealing deeper insight into the circuit design and built databases of the blue-print of the cortical column. In 2005 he used these databases to launched the Blue Brain Project. The BBP used IBMs most advanced supercomputers to reconstruct a detailed computer model of the neocortical column composed of 10000 neurons, more than 340 different types of neurons distributed according to a layer-based recipe of composition and interconnected with 30 million synapses (6 different types) according to synaptic mapping recipes. The Blue Brain team built dozens of applications that now allow automated reconstruction, simulation, visualization, analysis and calibration of detailed microcircuits. This Proof of Concept completed, Markrams lab has now set the agenda towards whole brain and molecular modeling.
With an in depth understanding of the neocortical microcircuit, Markram set a path to determine how the neocortex changes in Autism. He found hyper-reactivity due to hyper-connectivity in the circuitry and hyper-plasticity due to hyper-NMDA expression. Similar findings in the Amygdala together with behavioral evidence that the animal model of autism expressed hyper-fear led to the novel theory of Autism called the Intense World Syndrome proposed by Henry and Kamila Markram. The Intense World Syndrome claims that the brain of an Autist is hyper-sensitive and hyper-plastic which renders the world painfully intense and the brain overly autonomous. The theory is acquiring rapid recognition and many new studies have extended the findings to other brain regions and to other models of autism.
Markram aims to eventually build detailed computer models of brains of mammals to pioneer simulation-based research in the neuroscience which could serve to aggregate, integrate, unify and validate our knowledge of the brain and to use such a facility as a new tool to explore the emergence of intelligence and higher cognitive functions in the brain, and explore hypotheses of diseases as well as treatments.
Pascal FrossardTeaching & PhD PhD Students Yiming Qin, Alba Carballo Castro, Elisa Messori, Ke Wang, Sevda Ögüt, Tuna Alikasifoglu, William Cappelletti, Manuel Madeira, Mahdi Amiri, Ortal Yona Senouf, Vasiliki Rizou, Cem Bilaloglu, Abdellah Rahmani, Amel Abdelraheem, Jérémy Jean Philippe Baffou, Adam Hazimeh, Vincent Jung, Yike Zhao Past EPFL PhD Students Dan Jurca, Jean-Paul Wagner, Ivana Radulovic, Ivana Tosic, Effrosyni Kokiopoulou, Zafer Arican, Vijayaraghavan Thirumalai, Tamara Tosic, Eirina Bourtsoulatze, Elif Vur
Sylvain CalinonExpertise
robot learning, human-robot collaboration, optimal control, model-based optimization
I am a Senior Research Scientist at the Idiap Research Institute and a Lecturer at EPFL, with research interests covering robot learning, optimal control, geometrical approaches, and human-robot collaboration. My work focuses on human-centered robotics applications in which the robots can acquire new skills from only few demonstrations and interactions. It requires the development of models that can exploit the st
Alexander MathisAlexander studied pure mathematics with a minor in logic and theory of science at the Ludwig Maximilians University in Munich. For his PhD also at LMU, he worked on optimal coding approaches to elucidate the properties of grid cells. As a postdoctoral fellow with Prof. Venkatesh N. Murthy at Harvard University and Prof. Matthias Bethge at Tuebingen AI, he decided to study olfactory behaviors such as odor-guided navigation, social behaviors and the cocktail party problem in mice. During this time, he increasingly got interested sensorimotor behaviors beyond olfaction and started working on proprioception, motor adaption, as well as computer vision tools for measuring animal behavior.
In his group, he is interested in elucidating how the brain gives rise to adaptive behavior. One of the major goals is to synthesize large datasets into computationally useful information. For those purposes, he develops algorithms and systems to analyze animal behavior (e.g. DeepLabCut), neural data, as well as creates experimentally testable computational models.
Antoine BosselutAntoine Bosselut is an assistant professor at EPFL. He leads the EPFL NLP group, which conducts research on natural language processing (NLP) systems that can model, represent, and reason about human and world knowledge.
Prior to joining EPFL, he was a postdoctoral researcher at Stanford University working in the
SNAP
and
NLP
groups and a young investigator on the
Mosaic
project at the
Allen Institute for AI
. He completed his PhD at the University of Washington.
Michaël UnserExpertise
Image Processing, Medical Imaging, Biological Imaging, Wavelets, Splines, Multiresolution
Michael Unser is Full Professor with EPFL’s school of Engineering and the academic director of EPFL's Center for Imaging, Lausanne, Switzerland. His primary areas of investigation are signal processing, biomedical imaging and applied functional analysis. He is internationally recognized for his research contributions to sampling theory, wavelets, the use of splines for image processing, stochastic processes,
Francesco MondadaDr. Mondada received his M.Sc. in micro-engineering in 1991 and his Doctoral degree in 1997 at EPFL. During his thesis he co-founded the company K-Team, being both CEO and president of the company for about 5 years. He is one of the three main developers of the Khepera robot, considered as a standard in bio-inspired robotics and used by more than 1,000 universities and research centers worldwide. Fully back in research in 2000 and after a short period at CALTECH, he participated to the SWARM-BOTS project as the main developer of the s-bot robot platform, which was ranked on position 39 in the list of The 50 Best Robots Ever (fiction or real) by the Wired Journal in 2006. The SWARM-BOTS project was selected as FET-IST success story by the EU commission. He is author of more than 100 papers in the field of bio-inspired robotics and system level robot design. He is co-editor of several international conference proceedings. In November 2005 he received the EPFL Latsis University prize for his contributions to bio-inspired robotics. In 2011 he received the "Crédit Suisse Award for Best Teaching" from EPFL and in 2012 the "polysphère" award from the students as best teacher in the school of engineering. His interests include the development of innovative mechatronic solutions for mobile and modular robots, the creation of know-how for future embedded applications, and making robot platforms more accessible for education, research, and industrial development.
Martin JaggiMartin Jaggi is a Tenure Track Assistant Professor at EPFL, heading the Machine Learning and Optimization Laboratory. Before that, he was a post-doctoral researcher at ETH Zurich, at the Simons Institute in Berkeley, and at École Polytechnique in Paris. He has earned his PhD in Machine Learning and Optimization from ETH Zurich in 2011, and a MSc in Mathematics also from ETH Zurich.
Marion LeleuExpertise
Bioinformatics, Genomics, High-Throughput Sequencing, Data Mining
Mission
Provide advanced and personalised support and consulting in bioinformatics and biostatistics for the scientific community of EPFL, UNIL, and other Institutes.Her main areas of interest and expertise include but are not limited to, genomics, transcriptomics and epigenetics, with a strong interest in integrating multiple modalities, and choosing the appropriate statistical models, to gain a comprehensive understanding of compl
Jan Sickmann HesthavenProf. Hesthaven received an M.Sc. in computational physics from the Technical University of Denmark (DTU) in August 1991. During the studies, the last 6 months of 1989 was spend at JET, the european fusion laboratory in Culham, UK. Following graduation, he was awarded a 3 year fellowship to begin work towards a Ph.D. at Riso National Laboratory in the Department of Optics and Fluid Dynamics. During the 3 years of study, the academic year of 1993-1994 was spend in the Division of Applied Mathematics at Brown University and three 3 months during the summer of 1994 in Department of Mathematics and Statistics at University of New Mexico. In August 1995, he recieved a Ph.D. in Numerical Analysis from the Institute of Mathematical Modelling (DTU). Following graduation in August 1995, he was awarded an NSF Postdoctoral Fellowship in Advanced Scientific Computing and was approinted Visiting Assistant Professor in the Division of Applied Mathematics at Brown University. In December of 1996, he was appointed consultant to the Institute of Computer Applications in Science and Engineering(ICASE) at NASA Langley Research Center (NASA LaRC). As of July 1999, he was appointed Assistant Professor of Applied Mathematics, in September 2000 he was awarded an Alfred P. Sloan Fellowship, as of July 2001 he was awarded a Manning Assistant Professorship, and in March 2002, he was awarded an NSF Career Award. In January 2003, he was promoted to Associate Professor of Applied Mathematics with tenure and in May 2004 he was awarded Philip J. Bray Award for Excellence in Teaching in the Sciences (the highest award given for teaching excellence in all sciences at Brown University). He was promoted to Professor of Applied Mathematics as of July 2005. From October 2006 to June 2013, he was the Founding Director of the Center for Computation and Visualization (CCV) at Brown University. As of October 2007, he holds the (honorary) title of Professor (Adjunct) at the Technical University of Denmark. In November 2009, he successfully defended his dr.techn thesis at the Technical University of Denmark and was rewarded the degree of Doctor Technices -- the highest academic distinction awarded based on ... substantial and lasting contributions that has helped to move the research area forward and penetrated into applications. As grant Co-PI he served from Aug 2010 to June 2013 as Deputy Director of the Institute of Computational and Experimental Research in Mathematics (ICERM), the newest NSF Mathematical Sciences Research Institute. After having spend his entire academic career at Brown University, Prof Hesthaven decided to pursue new challenges and joined the Mathematics Institute of Computational Science and Engineering (MATHICSE) at Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland in July 2013. In March 2014 he was elected SIAM Fellow for contributions to high-order methods for partial differential equations.
Mario PaoloneMario Paolone received the M.Sc. (with honors) and the Ph.D. degree in electrical engineering from the University of Bologna, Italy, in 1998 and 2002, respectively. In 2005, he was appointed assistant professor in power systems at the University of Bologna where he was with the Power Systems laboratory until 2011. In 2010, he received the Associate Professor eligibility from the Politecnico di Milano, Italy. Since 2011 he joined the Swiss Federal Institute of Technology, Lausanne, Switzerland, where he is now Full Professor, Chair of the Distributed Electrical Systems laboratory and Head of the Swiss Competence Center for Energy Research (SCCER) FURIES (Future Swiss Electrical infrastructure). He was co-chairperson of the technical programme committees of the 9th edition of the International Conference of Power Systems Transients (IPST 2009) and of the 2016 Power Systems Computation Conference (PSCC 2016). He was chair of the technical programme committee of the 2018 Power Systems Computation Conference (PSCC 2018). In 2013, he was the recipient of the IEEE EMC Society Technical Achievement Award. He was co-author of several papers that received the following awards: best IEEE Transactions on EMC paper award for the year 2017, in 2014 best paper award at the 13th International Conference on Probabilistic Methods Applied to Power Systems, Durham, UK, in 2013 Basil Papadias best paper award at the 2013 IEEE PowerTech, Grenoble, France, in 2008 best paper award at the International Universities Power Engineering Conference (UPEC). He was the founder Editor-in-Chief of the Elsevier journal Sustainable Energy, Grids and Networks and was Associate Editor of the IEEE Transactions on Industrial Informatics. His research interests are in power systems with particular reference to real-time monitoring and operation, power system protections, power systems dynamics and power system transients. Mario Paolone is author or coauthor of over 300 scientific papers published in reviewed journals and international conferences.
