Transformer Architectures: Subquadratic Attention Mechanisms

Related lectures (48)

Discusses matrix multiplication using divide-and-conquer techniques and introduces Strassen's algorithm for improved efficiency.

Covers the time complexity of algorithms and includes a quiz.

Explores properties of 3x3 matrices with real coefficients and determinant calculation methods.

Covers matrix notation, arithmetic, multiplication, algorithms, and complexity of matrix multiplication.

Introduces key concepts in linear algebra, including matrices, operations, and numerical invariants.

Covers algorithmic complexity and travel time analysis, focusing on measuring the time taken by algorithms and evaluating their performance.

Explains square matrices, including invertible, triangular, and diagonal matrices, and their properties.

Covers generative models with a focus on self-attention and transformers, discussing sampling methods and empirical means.

Covers matrix notation, arithmetic, multiplication algorithms, and complexity analysis.

Discusses Strassen's algorithm for matrix multiplication and heaps, covering efficient algorithms and their applications in computer science.

Explores the complexity of matrix computations, focusing on low rank approximation algorithms and their applications.

Covers the definitions and properties of matrices, including matrix operations and determinants.

Explores theoretical and practical aspects of fast matrix-matrix multiplication algorithms and their significance in computer science.

Explores the properties of invertible matrices, including unique solutions and linear independence.

Explores the equivalence between different properties of linear transformations represented by matrices and various matrix operations.

Explores upper triangular block matrices, invertibility, elementary transformations, and determinants with examples.

Explores the Divide-and-Conquer algorithm for matrix multiplication, including Strassen's Method and its significance in optimizing time complexity.

Covers matrix operations, including multiplication, transposition, powers, and inverses, and explains how to determine if a matrix is invertible.

Covers matrix product, inverses, and properties of invertible matrices.

Covers the fundamentals of linear equations, matrices, and systems of linear equations, including matrix operations and solutions.

Page 1 of 3