Lecture

Neural Network Syntactic Parsing

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Description

This lecture covers neural network-based syntactic parsing, focusing on graph-based parsing with iterative refinement. It discusses LSTM-based dependency score estimation, a neural graph-based dependency parser, and the G2GT iterative refinement parser. The lecture explains the EPEL motivation for finding graph structures, the RNGTr encoder architecture, and the iterative non-autoregressive refinement process. It introduces the Recursive Non-Autoregressive Graph-to-Graph Transformer (RNGTr) model, detailing the encoder and score function. The lecture also explores model selection, results on UD and Penn Treebanks, and the performance analysis based on dependency types and projectivity of dependency graphs.

This video is available exclusively on Mediaspace for a restricted audience. Please log in to MediaSpace to access it if you have the necessary permissions.

Watch on Mediaspace

Official source

Related concepts (32)

Parsing

Parsing, syntax analysis, or syntactic analysis is the process of analyzing a string of symbols, either in natural language, computer languages or data structures, conforming to the rules of a formal grammar. The term parsing comes from Latin pars (orationis), meaning part (of speech). The term has slightly different meanings in different branches of linguistics and computer science. Traditional sentence parsing is often performed as a method of understanding the exact meaning of a sentence or word, sometimes with the aid of devices such as sentence diagrams.

Parse tree

A parse tree or parsing tree or derivation tree or concrete syntax tree is an ordered, rooted tree that represents the syntactic structure of a string according to some context-free grammar. The term parse tree itself is used primarily in computational linguistics; in theoretical syntax, the term syntax tree is more common. Concrete syntax trees reflect the syntax of the input language, making them distinct from the abstract syntax trees used in computer programming.

Dependency grammar

Dependency grammar (DG) is a class of modern grammatical theories that are all based on the dependency relation (as opposed to the constituency relation of phrase structure) and that can be traced back primarily to the work of Lucien Tesnière. Dependency is the notion that linguistic units, e.g. words, are connected to each other by directed links. The (finite) verb is taken to be the structural center of clause structure. All other syntactic units (words) are either directly or indirectly connected to the verb in terms of the directed links, which are called dependencies.

Recurrent neural network

A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.

Treebank

In linguistics, a treebank is a parsed text corpus that annotates syntactic or semantic sentence structure. The construction of parsed corpora in the early 1990s revolutionized computational linguistics, which benefitted from large-scale empirical data. The term treebank was coined by linguist Geoffrey Leech in the 1980s, by analogy to other repositories such as a seedbank or bloodbank. This is because both syntactic and semantic structure are commonly represented compositionally as a tree structure.

Related lectures (64)

Syntactic Parsing and Graph2Graph Transformers

Covers syntactic parsing, dependency grammar, and graph-based neural network parsing.

Language Modelling and Recurrent Neural Networks

Explores language modelling, RNNs, n-gram models, LSTMs, and bidirectional RNNs.

Sequence to Sequence Models: Overview and Applications

Covers sequence to sequence models, their architecture, applications, and the role of attention mechanisms in improving performance.

Recurrent Neural Networks: LSTMs & GRUs

Explores LSTM and GRU variants in recurrent neural networks, addressing challenges and advantages.

Recurrent Neural Networks: Training and Challenges