Concept

Deep belief network

In machine learning, a deep belief network (DBN) is a generative graphical model, or alternatively a class of deep neural network, composed of multiple layers of latent variables ("hidden units"), with connections between the layers but not between units within each layer. When trained on a set of examples without supervision, a DBN can learn to probabilistically reconstruct its inputs. The layers then act as feature detectors. After this learning step, a DBN can be further trained with supervision to perform classification. DBNs can be viewed as a composition of simple, unsupervised networks such as restricted Boltzmann machines (RBMs) or autoencoders, where each sub-network's hidden layer serves as the visible layer for the next. An RBM is an undirected, generative energy-based model with a "visible" input layer and a hidden layer and connections between but not within layers. This composition leads to a fast, layer-by-layer unsupervised training procedure, where contrastive divergence is applied to each sub-network in turn, starting from the "lowest" pair of layers (the lowest visible layer is a training set). The observation that DBNs can be trained greedily, one layer at a time, led to one of the first effective deep learning algorithms. Overall, there are many attractive implementations and uses of DBNs in real-life applications and scenarios (e.g., electroencephalography, drug discovery). The training method for RBMs proposed by Geoffrey Hinton for use with training "Product of Experts" models is called contrastive divergence (CD). CD provides an approximation to the maximum likelihood method that would ideally be applied for learning the weights. In training a single RBM, weight updates are performed with gradient descent via the following equation: where, is the probability of a visible vector, which is given by . is the partition function (used for normalizing) and is the energy function assigned to the state of the network. A lower energy indicates the network is in a more "desirable" configuration.

Official source

https://en.wikipedia.org/wiki/Deep_belief_network

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related courses (14)

EE-556: Mathematics of data: from theory to computation

This course provides an overview of key advances in continuous optimization and statistical analysis for machine learning. We review recent learning formulations and models as well as their guarantees

PHYS-467: Machine learning for physicists

Machine learning and data analysis are becoming increasingly central in sciences including physics. In this course, fundamental principles and methods of machine learning will be introduced and practised. ...

PHYS-754: Lecture series on scientific machine learning

This lecture presents ongoing work on how scientific questions can be tackled using machine learning. Machine learning enables extracting knowledge from data computationally and in an automatized way. We will learn on examples how this is influencing the v ...

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Official source

https://en.wikipedia.org/wiki/Deep_belief_network

About this result

Related courses (14)

EE-556: Mathematics of data: from theory to computation

PHYS-467: Machine learning for physicists

PHYS-754: Lecture series on scientific machine learning

Related lectures (32)

Generative Modeling: Then, Now and Tomorrow?

Explores the evolution of generative modeling, from traditional methods to cutting-edge advancements, addressing challenges and envisioning future possibilities.

The Problem of Overfitting

Discusses the problem of overfitting in deep networks and the importance of controlling flexibility to avoid it.

Artificial Neural Networks and Deep Learning: Loss landscape and optimization methods

Explores error function landscape, optimization methods, and deep neural networks for classification.

Related publications (26)

Feature Learning with Gradient-Based Methods: Insights from Solvable Models of Neural Networks

Understanding why gradient-trained neural networks succeed requires theoretical models that capture their ability to learn internal representations from high-dimensional data. Despite the empirical dominance of large-scale deep learning, even small neural networks remain analytically elusive: their learning dynamics and representational structure are not yet fully understood. This thesis develops solvable models of shallow neural networks in high dimensions, providing a rigorous framework for feature learni

EPFL2026

Provable Learning of Random Hierarchy Models and Hierarchical Shallow-to-Deep Chaining

The empirical success of deep learning is often attributed to deep networks’ ability to exploit hierarchical structure in data, constructing increasingly complex features across layers. Yet despite substantial progress in deep learning theory, most optimization results still focus on networks with only two or three layers, leaving the theoretical understanding of hierarchical learning in genuinely deep models limited. This leads to a natural question: can we prove that deep networks, trained with gradient-b

ML Research Press2026

Instance-level quantitative saliency in multiple sclerosis lesion segmentation

Meritxell Bach Cuadra

In recent years, explainable methods for artificial intelligence (XAI) have tried to reveal and describe models’ decision mechanisms in the case of classification and even for segmentation. However, XAI methods for semantic segmentation and in particular for single specific instances (e.g. one given lesion among others of the same class in medical imaging) have yet to be developed to understand what drove the detection and contouring of the latter, which is crucial for all multi-lesional diseases. We propos

Nature Research2026

Related people (32)

Sébastien Marcel

Expertise image processing, pattern recognition, machine learning, biometrics Current work Face recognitionSpeaker recognitionMobile biometrics ( http://www.mobioproject.org )Spoofing ( http://www.tabularasa-euproject.org )Biometrics evaluation and testing ( http://www.beat-eu.org ) Sébastien Marcel is the head of the Biometrics group and senior research scientist at the Idiap Research Institute (CH), where he leads a research team and conducts research on biometrics and more particularly face recognition,

Pascal Frossard

Teaching & PhD PhD Students Yiming Qin, Alba Carballo Castro, Elisa Messori, Ke Wang, Sevda Ögüt, Tuna Alikasifoglu, William Cappelletti, Manuel Madeira, Mahdi Amiri, Ortal Yona Senouf, Vasiliki Rizou, Cem Bilaloglu, Abdellah Rahmani, Amel Abdelraheem, Jérémy Jean Philippe Baffou, Adam Hazimeh, Vincent Jung, Yike Zhao Past EPFL PhD Students Dan Jurca, Jean-Paul Wagner, Ivana Radulovic, Ivana Tosic, Effrosyni Kokiopoulou, Zafer Arican, Vijayaraghavan Thirumalai, Tamara Tosic, Eirina Bourtsoulatze, Elif Vur

Hervé Bourlard

Teaching & PhD Past EPFL PhD Students Astrid Hagen, Katrin Weber, Todd Stephenson, Jitendra Ajmera, Shajith Ikbal, Mathew Magimai Doss, Mohamed Faouzi Benzeghiba, Mark Barnard, Yann Rodriguez, Hemant Misra, Christos Dimitrakakis, Guillaume Lathoud, Alexei Pozdnoukhov, Agnès Just, Mikaela Keller, Norman Hoon Thian Poh, Dong Zhang, Florent Monay Michaud, Pedro Manuel Da Silva Quelhas, Kevin Smith, Silèye Oumar Ba, Bertrand Mesot, Hamed Ketabdar, David Grangier, Guillermo Aradilla, Jean-François Paiement, Nico

Joachim Ludovic Hugonot

Kasper Smeets

Alexandre Schmid

Alexandre Schmid received the M.Sc. degree in microengineering and the Ph.D. degree in electrical engineering from the Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland, in 1994 and 2000, respectively. Since 1994, he has been with the EPFL, working with the Integrated Systems Laboratory as a Research and Teaching Assistant, and with the Electronics Laboratories as a Postdoctoral Fellow. In 2002, he was a Senior Research Associate with the Microelectronic Systems Laboratory, where he has been conducting research in the fields of bioelectronic interfaces and implantable biomedical electronics, nonconventional signal processing and neuromorphic hardware, and reliability of nanoelectronic devices, and also teaches with the Microengineering and Electrical Engineering Departments of EPFL. Since 2011, he is a Maître d'Enseignement et de Recherche (MER) Faculty Member with EPFL. He is a coauthor of two books, Reliability of Nanoscale Circuits and Systems, Methodologies and Circuit Architectures, Springer, 2011, and Wireless Cortical Implantable Systems, Springer, 2013, and a coeditor of one book, as well as over 100 articles published in journals and conferences. Dr. Schmid has served as the General Chair of the Fourth International Conference on Nano-Networks in 2009 and has been serving as an Associate Editor of the Institute of Electrical, Information, and Communication Engineers Electronics Express since 2009.

Pascal Fua

Pascal Fua received an engineering degree from Ecole Polytechnique, Paris, in 1984 and the Ph.D. degree in Computer Science from the University of Orsay in 1989. He then worked at SRI International and INRIA Sophia-Antipolis as a Computer Scientist. He joined EPFL in 1996 where he is now a Professor in the School of Computer and Communication Science and heads the Computer Vision Laboratory. His research interests include shape modeling and motion recovery from images, analysis of microscopy images, and Augmented Reality. His research interests include shape modeling and motion recovery from images, analysis of microscopy images, and machine learning. He has (co)authored over 300 publications in refereed journals and conferences. He is an IEEE Fellow and has been an Associate Editor of IEEE journal Transactions for Pattern Analysis and Machine Intelligence. He often serves as program committee member, area chair, and program chair of major vision conferences and has cofounded three spinoff companies (Pix4D, PlayfulVision, and NeuralConcept).

Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

Adélie Eliane Garin

Teaching & PhD Courses Sustainability ENV-101 This course presents global sustainability issues. It introduces planetary boundaries, their systemic interconnections and social justice issues. The interdisciplinary approach introduces group work and systems thinking (interdependencies in a complex system). Sustainability in my research ENG-650 This course is designed to develop the necessary transversal skills to foster sustainability integration in the present and future rese

Kathryn Hess Bellwald

Kathryn Hess Bellwald received her PhD from MIT in 1989 and held positions at the universities of Stockholm, Nice, and Toronto before moving to the EPFL.Her research focuses on algebraic topology and its applications, primarily in the life sciences, but also in materials science. She has published extensively on topics in pure algebraic topology including homotopy theory, operad theory, and algebraic K-theory. On the applied side, she has elaborated methods based on topological data analysis for high-throughput screening of nanoporous crystalline materials, classification and synthesis of neuron morphologies, and classification of neuronal network dynamics. She has also developed and applied innovative topological approaches to network theory, leading to a powerful, parameter-free mathematical framework relating the activity of a neural network to its underlying structure, both locally and globally.In 2016 she was elected to Swiss Academy of Engineering Sciences and was named a fellow of the American Mathematical Society and a distinguished speaker of the European Mathematical Society in 2017. In 2021 she gave an invited Public Lecture at the European Congress of Mathematicians. She has won several teaching prizes at EPFL, including the Crédit Suisse teaching prize and the Polysphère d’Or.

Related units (9)

Computer Vision Laboratory

SDSC - Administration

Signal Processing Laboratory 4

Related concepts (3)

Feature learning

In machine learning, feature learning or representation learning is a set of techniques that allows a system to automatically discover the representations needed for feature detection or classification from raw data. This replaces manual feature engineering and allows a machine to both learn the features and use them to perform a specific task. Feature learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process.

Generative model

In statistical classification, two main approaches are called the generative approach and the discriminative approach. These compute classifiers by different approaches, differing in the degree of statistical modelling. Terminology is inconsistent, but three major types can be distinguished, following : A generative model is a statistical model of the joint probability distribution on given observable variable X and target variable Y; A discriminative model is a model of the conditional probability of the target Y, given an observation x; and Classifiers computed without using a probability model are also referred to loosely as "discriminative".

Convolutional neural network

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.