Lectures related to Second-Order Model Compression

Provable Benefits of Overparameterization in Model Compression

Explores the provable benefits of overparameterization in model compression, emphasizing the efficiency of deep neural networks and the importance of retraining for improved performance.

Model Compression: Techniques for Efficient NLP Models

Explores model compression techniques in NLP, discussing pruning, quantization, weight factorization, knowledge distillation, and attention mechanisms.

Model Compression Techniques: Enhancing Neural Networks

Covers model compression techniques to enhance the efficiency of large language models in production settings.

Dimensionality Reduction: PCA and Autoencoders

Introduces artificial neural networks, CNNs, and dimensionality reduction using PCA and autoencoders.

Convolutional Neural Networks

Covers Convolutional Neural Networks, including layers, training strategies, standard architectures, tasks like semantic segmentation, and deep learning tricks.

Language Models: Fixed-context and Recurrent Neural Networks

Discusses language models, focusing on fixed-context neural models and recurrent neural networks.

Deep Neural Networks: Optimization and Approximation

Explores optimization and approximation in deep neural networks, including optimal control and numerical experiments.

Scaling Language Models: Efficiency and Deployment

Covers the scaling of language models, focusing on training efficiency and deployment considerations.

Non-Conceptual Knowledge Systems

Delves into the impact of deep learning on non-conceptual knowledge systems and the advancements in transformers and generative adversarial networks.

Nonlinear Supervised Learning

Explores the inductive bias of different nonlinear supervised learning methods and the challenges of hyper-parameter tuning.

Deep Learning Fundamentals Log in to Mediaspace to watch this video

Introduces deep learning fundamentals, covering data representations, neural networks, and convolutional neural networks.

Neural Networks for NLP Log in to Mediaspace to watch this video

Covers modern Neural Network approaches to NLP, focusing on word embeddings, Neural Networks for NLP tasks, and future Transfer Learning techniques.

Auto-encoder and GANs Log in to Mediaspace to watch this video

Covers auto-encoders for data compression and GANs for data generation.

Deep Learning: Convolutional Networks Log in to Mediaspace to watch this video

Explores convolutional neural networks, backpropagation, and stochastic gradient descent in deep learning.

Optical Computing: All-Analog Photoelectronic Chip for Vision Tasks Log in to Mediaspace to watch this video

Presents an all-analog photoelectronic chip for high-speed vision tasks, addressing challenges in classical computation and proposing a hybrid optical-electrical framework.

Neural Networks: Perceptron Model and Backpropagation Algorithm Log in to Mediaspace to watch this video

Covers the perceptron model and backpropagation algorithm in neural networks.

Feed-forward Networks Log in to Mediaspace to watch this video

Introduces feed-forward networks, covering neural network structure, training, activation functions, and optimization, with applications in forecasting and finance.

Deep Learning Fundamentals Log in to Mediaspace to watch this video

Introduces deep learning, from logistic regression to neural networks, emphasizing the need for handling non-linearly separable data.

Deep Learning: Convolutional Neural Networks and Training Techniques Log in to Mediaspace to watch this video

Discusses convolutional neural networks, their architecture, training techniques, and challenges like adversarial examples in deep learning.

Learning from the Interconnected World with Graphs Log in to Mediaspace to watch this video

Explores learning from interconnected data using graphs, covering challenges, GNN design, research landscapes, and democratization of Graph ML.