Publication

A multimodal approach to extract optimized audio features for speaker detection

Jean-Philippe Thiran, Murat Kunt, Torsten Butz
2005
Conference Papers

Abstract

We present a method that exploits an information theoretic framework to extract optimal audio features with respect to the video features. A simple measure of mutual information between the resulting audio features and the video ones allows to detect the active speaker among different candidates. The results show that our method is able to exploit the shared speech information contained in audio and video signals to recover their common source.

Official source

https://infoscience.epfl.ch/entities/publication/a2c420eb-0c0f-43c9-b090-be36766604bd

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Jean-Philippe Thiran, Murat Kunt, Torsten Butz
2005
Conference Papers

Abstract

Official source

https://infoscience.epfl.ch/entities/publication/a2c420eb-0c0f-43c9-b090-be36766604bd

About this result

Related concepts (32)

Video

Video is an electronic medium for the recording, copying, playback, broadcasting, and display of moving media. Video was first developed for mechanical television systems, which were quickly replaced by cathode-ray tube (CRT) systems which, in turn, were replaced by flat panel displays of several types. Video systems vary in display resolution, aspect ratio, refresh rate, color capabilities and other qualities. Analog and digital variants exist and can be carried on a variety of media, including radio broadcasts, magnetic tape, optical discs, , and network streaming.

Mutual information

In probability theory and information theory, the mutual information (MI) of two random variables is a measure of the mutual dependence between the two variables. More specifically, it quantifies the "amount of information" (in units such as shannons (bits), nats or hartleys) obtained about one random variable by observing the other random variable. The concept of mutual information is intimately linked to that of entropy of a random variable, a fundamental notion in information theory that quantifies the expected "amount of information" held in a random variable.

Interaction information

The interaction information is a generalization of the mutual information for more than two variables. There are many names for interaction information, including amount of information, information correlation, co-information, and simply mutual information. Interaction information expresses the amount of information (redundancy or synergy) bound up in a set of variables, beyond that which is present in any subset of those variables. Unlike the mutual information, the interaction information can be either positive or negative.

Related publications (3)

A multimodal approach to extract optimized audio features for speaker detection

Chat with Graph Search

Redundant information encoding in primary motor cortex during natural and prosthetic motor control

Joint Smoothing and Source Rate Selection for Guaranteed Service Networks

Interpolative multiresolution coding of advanced television with compatible subchannels