Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Chemical Reactions: Transformer Architecture

In course

PHYS-754: Lecture series on scientific machine learning

This lecture presents ongoing work on how scientific questions can be tackled using machine learning. Machine learning enables extracting knowledge from data computationally and in an automatized way. We will learn on examples how this is influencing the v ...

Description

This lecture covers the concept of atom mapping in chemical reactions, the transformer architecture, and the transition from the Molecular Transformer to reaction grammar. It discusses the dataset of reaction SMILES, the discovery and utilization of transformers in capturing the hidden grammar of reactions, and the comparison with other tools in terms of correctness and performance.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Instructors (10)

Giuseppe Carleo is a computational quantum physicist, whose main focus is the development of advanced numerical algorithms tostudy challenging problems involving strongly interacting quantum systems.He is best known for the introduction of machine learning techniques to study both equilibrium and dynamical properties,based on a neural-network representations of quantum states, as well for the time-dependent variational Monte Carlo method.He earned a Ph.D. in Condensed Matter Theory from the International School for Advanced Studies (SISSA) in Italy in 2011.He held postdoctoral positions at the Institut d’Optique in France and ETH Zurich in Switzerland, where he alsoserved as a lecturer in computational quantum physics.In 2018, he joined the Flatiron Institute in New York City in 2018 at the Center for Computational Quantum Physics (CCQ), working as a Research Scientist and project leader, and also leading the development of the open-source project NetKet.Since September 2020 he is an assistant professor at EPFL, in Switzerland, leading the Computational Quantum Science Laboratory (CQSL).

Paolo De Los Rios

Paolo De Los Rios earned his master in Electronic Engineering at the Turin Institute of Technology (Politecnico di Torino) in May 1993. In November 1993 he moved to Trieste, Italy, to enter the PhD program in Theoretical Condensed Matter Theory at the International School for Advanced Studies (SISSA/ISAS) where he obtained the PhD degree in October 1996 for his work on the statistical physics of disordered systems. After a one year postdoc at the Max-Planck Institute for the Physics of Complex Systems in Dresden, Germany, in November 1997 he moved to the University of Fribourg, Switzerland, to join the group of Prof. Yi-Cheng Zhang. There he has worked on various applications of statistical physics to complex systems. In September 2000 he has been appointed Assistant Professor in Statistical Physics of Living Matter and Complex Systems at the Institute of Theoretical Physics of the University of Lausanne, Switzerland. Since April 2010 he is Associate Professor at the Institute of Theoretical Physics of the Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland.

Michele Ceriotti

Official source

Related concepts (32)

Unsupervised learning

Unsupervised learning, is paradigm in machine learning where, in contrast to supervised learning and semi-supervised learning, algorithms learn patterns exclusively from unlabeled data. Neural network tasks are often categorized as discriminative (recognition) or generative (imagination). Often but not always, discriminative tasks use supervised methods and generative tasks use unsupervised (see Venn diagram); however, the separation is very hazy. For example, object recognition favors supervised learning but unsupervised learning can also cluster objects into groups.

Transformer (machine learning model)

A transformer is a deep learning architecture that relies on the parallel multi-head attention mechanism. The modern transformer was proposed in the 2017 paper titled 'Attention Is All You Need' by Ashish Vaswani et al., Google Brain team. It is notable for requiring less training time than previous recurrent neural architectures, such as long short-term memory (LSTM), and its later variation has been prevalently adopted for training large language models on large (language) datasets, such as the Wikipedia corpus and Common Crawl, by virtue of the parallelized processing of input sequence.

Feature learning

In machine learning, feature learning or representation learning is a set of techniques that allows a system to automatically discover the representations needed for feature detection or classification from raw data. This replaces manual feature engineering and allows a machine to both learn the features and use them to perform a specific task. Feature learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process.

Self-supervised learning

Self-supervised learning (SSL) is a paradigm in machine learning for processing data of lower quality, rather than improving ultimate outcomes. Self-supervised learning more closely imitates the way humans learn to classify objects. The typical SSL method is based on an artificial neural network or other model such as a decision list. The model learns in two steps. First, the task is solved based on an auxiliary or pretext classification task using pseudo-labels which help to initialize the model parameters.

Machine learning

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. Recently, generative artificial neural networks have been able to surpass results of many previous approaches.

Related lectures (53)

Language Models: From Theory to Computation

Explores the mathematics of language models, covering architecture design, pre-training, and fine-tuning, emphasizing the importance of pre-training and fine-tuning for various tasks.

Introduction to Machine Learning

Provides an overview of Machine Learning, including historical context, key tasks, and real-world applications.

Nonlinear Supervised Learning

Explores the inductive bias of different nonlinear supervised learning methods and the challenges of hyper-parameter tuning.

Non conceptual knowledge systems

Explores the impact of Deep learning on Digital Humanities, focusing on non conceptual knowledge systems and recent advancements in AI.

Image Representation Insights