Lecture

Multi-layered Perceptron: History and Training Algorithm

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

EE-311: Fundamentals of machine learning

Ce cours présente une vue générale des techniques d'apprentissage automatique, passant en revue les algorithmes, le formalisme théorique et les protocoles expérimentaux. ...

Description

This lecture covers the historical development of artificial neural networks, starting with the threshold logic unit and the perceptron. The instructor explains the training algorithm for the perceptron, focusing on the gradient descent method. The lecture then introduces the multi-layered perceptron, discussing its architecture, activation functions, and the backpropagation algorithm. The importance of feature design and the limitations of linear models are also addressed. The instructor demonstrates how a multi-layered perceptron can approximate any continuous function and the challenges in interpreting its operations.

Instructors (2)

Michael Stefan Daniel Liebling

Teaching & PhD Past EPFL PhD Students as codirector Christian Jaques, Olivia Mariani, Adrian Shajkofci, François Thierry M Marelli Courses Fundamentals of machine learning EE-311 This course provides a general overview of machine learning, covering the main algorithms, theoretical formalisms, and experimental protocols.

François Fleuret

Official source

Ontological neighbourhood

Information engineering

Machine learning: Artificial neural networks

Related concepts (32)

Transformer (machine learning model)

A transformer is a deep learning architecture that relies on the parallel multi-head attention mechanism. The modern transformer was proposed in the 2017 paper titled 'Attention Is All You Need' by Ashish Vaswani et al., Google Brain team. It is notable for requiring less training time than previous recurrent neural architectures, such as long short-term memory (LSTM), and its later variation has been prevalently adopted for training large language models on large (language) datasets, such as the Wikipedia corpus and Common Crawl, by virtue of the parallelized processing of input sequence.

Feedforward neural network

A feedforward neural network (FNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. Its flow is uni-directional, meaning that the information in the model flows in only one direction—forward—from the input nodes, through the hidden nodes (if any) and to the output nodes, without any cycles or loops, in contrast to recurrent neural networks, which have a bi-directional flow.

Machine learning

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. Recently, generative artificial neural networks have been able to surpass results of many previous approaches.

Active learning (machine learning)

Active learning is a special case of machine learning in which a learning algorithm can interactively query a user (or some other information source) to label new data points with the desired outputs. In statistics literature, it is sometimes also called optimal experimental design. The information source is also called teacher or oracle. There are situations in which unlabeled data is abundant but manual labeling is expensive. In such a scenario, learning algorithms can actively query the user/teacher for labels.

Perceptron

In machine learning, the perceptron (or McCulloch-Pitts neuron) is an algorithm for supervised learning of binary classifiers. A binary classifier is a function which can decide whether or not an input, represented by a vector of numbers, belongs to some specific class. It is a type of linear classifier, i.e. a classification algorithm that makes its predictions based on a linear predictor function combining a set of weights with the feature vector.

Related lectures (64)

Deep Learning: Convolutional Neural Networks

Covers Convolutional Neural Networks, standard architectures, training techniques, and adversarial examples in deep learning.

Deep Learning Fundamentals

Introduces deep learning, from logistic regression to neural networks, emphasizing the need for handling non-linearly separable data.

Deep Learning: Multilayer Perceptron and Training

Covers deep learning fundamentals, focusing on multilayer perceptrons and their training processes.