Lecture

Building Physical Neural Networks

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

MSE-806: Spin-based devices for neuromorphic computing

The course entails a 5-days-program with lectures and exercises about spin-based computing and novel spin texture-based computing devices. An additional round table discussion and journal club session provide further knowledge about recent trends and chall ...

Description

This lecture explores the challenges of building physical neural networks, focusing on key aspects such as model size, density of connections, and trainability. The instructor discusses the need for depth in networks, long-range connections, and high connection density. The lecture delves into the limitations of physical systems in achieving these requirements and presents innovative solutions using spintronics and neuromorphic computing. The presentation covers topics like spintronics, neuromorphic computing, and the integration of emerging technologies for building complex neural networks.

Instructor

Dirk Grundler

Official source

Ontological neighbourhood

Information engineering

Machine learning: Artificial neural networks

Related concepts (32)

Reservoir computing

Reservoir computing is a framework for computation derived from recurrent neural network theory that maps input signals into higher dimensional computational spaces through the dynamics of a fixed, non-linear system called a reservoir. After the input signal is fed into the reservoir, which is treated as a "black box," a simple readout mechanism is trained to read the state of the reservoir and map it to the desired output. The first key benefit of this framework is that training is performed only at the readout stage, as the reservoir dynamics are fixed.

Types of artificial neural networks

There are many types of artificial neural networks (ANN). Artificial neural networks are computational models inspired by biological neural networks, and are used to approximate functions that are generally unknown. Particularly, they are inspired by the behaviour of neurons and the electrical signals they convey between input (such as from the eyes or nerve endings in the hand), processing, and output from the brain (such as reacting to light, touch, or heat). The way neurons semantically communicate is an area of ongoing research.

Backpropagation

As a machine-learning algorithm, backpropagation performs a backward pass to adjust the model's parameters, aiming to minimize the mean squared error (MSE). In a single-layered network, backpropagation uses the following steps: Traverse through the network from the input to the output by computing the hidden layers' output and the output layer. (the feedforward step) In the output layer, calculate the derivative of the cost function with respect to the input and the hidden layers.

Neuromorphic engineering

Neuromorphic computing is an approach to computing that is inspired by the structure and function of the human brain. A neuromorphic computer/chip is any device that uses physical artificial neurons to do computations. In recent times, the term neuromorphic has been used to describe analog, digital, mixed-mode analog/digital VLSI, and software systems that implement models of neural systems (for perception, motor control, or multisensory integration).

Physical neural network

A physical neural network is a type of artificial neural network in which an electrically adjustable material is used to emulate the function of a neural synapse or a higher-order (dendritic) neuron model. "Physical" neural network is used to emphasize the reliance on physical hardware used to emulate neurons as opposed to software-based approaches. More generally the term is applicable to other artificial neural networks in which a memristor or other electrically adjustable resistance material is used to emulate a neural synapse.

Related lectures (64)

Neuromorphic Computing: Concepts and Hardware Implementations

Covers neuromorphic computing, challenges in ternary and binary computing, hardware simulations of the brain, and new materials for artificial brain cells.

Deep Neural Networks: Optimization and Approximation

Explores optimization and approximation in deep neural networks, including optimal control and numerical experiments.

Neural Networks: Training and Activation

Explores neural networks, activation functions, backpropagation, and PyTorch implementation.

Splines and Machine Learning

Explores supervised learning as an ill-posed problem and the integration of sparse adaptive splines into neural architectures.

Deep Learning Fundamentals