Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Natural Language Generation

Description

This lecture covers the fundamentals of Natural Language Generation (NLG), including neural NLG models, decoding strategies, training algorithms, and evaluation methods. It delves into the challenges of exposure bias, biases in text generation models, and ethical considerations. The lecture emphasizes the importance of human evaluations and the need for safeguards to prevent harmful content generation. It also explores advanced topics such as model-based metrics, hidden biases, and the impact of large-scale language models on NLG research.

Official source

Related concepts (32)

Natural language generation

Natural language generation (NLG) is a software process that produces natural language output. A widely-cited survey of NLG methods describes NLG as "the subfield of artificial intelligence and computational linguistics that is concerned with the construction of computer systems than can produce understandable texts in English or other human languages from some underlying non-linguistic representation of information". While it is widely agreed that the output of any NLG process is text, there is some disagreement about whether the inputs of an NLG system need to be non-linguistic.

Language model

A language model is a probabilistic model of a natural language that can generate probabilities of a series of words, based on text corpora in one or multiple languages it was trained on. Large language models, as their most advanced form, are a combination of feedforward neural networks and transformers. They have superseded recurrent neural network-based models, which had previously superseded the pure statistical models, such as word n-gram language model.

Natural language processing

Natural language processing (NLP) is an interdisciplinary subfield of linguistics and computer science. It is primarily concerned with processing natural language datasets, such as text corpora or speech corpora, using either rule-based or probabilistic (i.e. statistical and, most recently, neural network-based) machine learning approaches. The goal is a computer capable of "understanding" the contents of documents, including the contextual nuances of the language within them.

Evaluation

In common usage, evaluation is a systematic determination and assessment of a subject's merit, worth and significance, using criteria governed by a set of standards. It can assist an organization, program, design, project or any other intervention or initiative to assess any aim, realisable concept/proposal, or any alternative, to help in decision-making; or to ascertain the degree of achievement or value in regard to the aim and objectives and results of any such action that has been completed.

Artificial neural network

Artificial neural networks (ANNs, also shortened to neural networks (NNs) or neural nets) are a branch of machine learning models that are built using principles of neuronal organization discovered by connectionism in the biological neural networks constituting animal brains. An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons.

Related lectures (50)

Natural Language Generation: Decoding & Training

Explores challenges in natural language generation, decoding algorithms, training issues, and reward functions.

Coreference Resolution

Delves into coreference resolution, discussing challenges, advancements, and evaluation methods.

Natural Language Generation: Decoding Techniques and Training Challenges

Covers decoding methods and training challenges in natural language generation.

Sequence to Sequence Models: Overview and Applications

Covers sequence to sequence models, their architecture, applications, and the role of attention mechanisms in improving performance.

Language Models: Fixed-context and Recurrent Neural Networks