Lecture

Introduction to Natural Language Processing

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture introduces the course on Natural Language Processing (NLP), covering the course overview, the definition of NLP, the worth of the course, applications, constraints, natural vs. formal languages, natural language functions, reasons why NLP is difficult, linguistic processing levels, and the history of NLP approaches. It delves into the evolution from traditional to modern NLP approaches, including statistical and machine learning, neural networks, and pre-trained large-scale language models. The lecture also discusses the importance of studying traditional NLP methods alongside modern ones, emphasizing the reuse of technology-independent methods. It concludes with an exploration of the challenges in NLP, such as power laws, resource constraints, and the multi-level nature of NLP processing.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Related concepts (32)

Natural language processing

Natural language processing (NLP) is an interdisciplinary subfield of linguistics and computer science. It is primarily concerned with processing natural language datasets, such as text corpora or speech corpora, using either rule-based or probabilistic (i.e. statistical and, most recently, neural network-based) machine learning approaches. The goal is a computer capable of "understanding" the contents of documents, including the contextual nuances of the language within them.

Language model

A language model is a probabilistic model of a natural language that can generate probabilities of a series of words, based on text corpora in one or multiple languages it was trained on. Large language models, as their most advanced form, are a combination of feedforward neural networks and transformers. They have superseded recurrent neural network-based models, which had previously superseded the pure statistical models, such as word n-gram language model.

Statistical model

A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form, the data-generating process. When referring specifically to probabilities, the corresponding term is probabilistic model. A statistical model is usually specified as a mathematical relationship between one or more random variables and other non-random variables.

Recurrent neural network

A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.

Statistical inference

Statistical inference is the process of using data analysis to infer properties of an underlying distribution of probability. Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population. Inferential statistics can be contrasted with descriptive statistics. Descriptive statistics is solely concerned with properties of the observed data, and it does not rest on the assumption that the data come from a larger population.

Related lectures (64)

Language Models: Fixed-context and Recurrent Neural Networks

Discusses language models, focusing on fixed-context neural models and recurrent neural networks.

Deep Learning for Question Answering

Explores deep learning for question answering, analyzing neural networks and model robustness to noise.

Neuro-symbolic Representations: Commonsense Knowledge & Reasoning

Delves into neuro-symbolic representations for commonsense knowledge and reasoning in natural language processing applications.

Introduction to Modern Natural Language Processing

Introduces the course on Modern Natural Language Processing, covering its significance, applications, challenges, and advancements in technology.

Coreference Resolution