Concept

Vector quantization

Vector quantization (VQ) is a classical quantization technique from signal processing that allows the modeling of probability density functions by the distribution of prototype vectors. It was originally used for data compression. It works by dividing a large set of points (vectors) into groups having approximately the same number of points closest to them. Each group is represented by its centroid point, as in k-means and some other clustering algorithms. The density matching property of vector quantization is powerful, especially for identifying the density of large and high-dimensional data. Since data points are represented by the index of their closest centroid, commonly occurring data have low error, and rare data high error. This is why VQ is suitable for lossy data compression. It can also be used for lossy data correction and density estimation. Vector quantization is based on the competitive learning paradigm, so it is closely related to the self-organizing map model and to sparse coding models used in deep learning algorithms such as autoencoder. The simplest training algorithm for vector quantization is: Pick a sample point at random Move the nearest quantization vector centroid towards this sample point, by a small fraction of the distance Repeat A more sophisticated algorithm reduces the bias in the density matching estimation, and ensures that all points are used, by including an extra sensitivity parameter : Increase each centroid's sensitivity by a small amount Pick a sample point at random For each quantization vector centroid , let denote the distance of and Find the centroid for which is the smallest Move towards by a small fraction of the distance Set to zero Repeat It is desirable to use a cooling schedule to produce convergence: see Simulated annealing. Another (simpler) method is LBG which is based on K-Means. The algorithm can be iteratively updated with 'live' data, rather than by picking random points from a data set, but this will introduce some bias if the data are temporally correlated over many samples.

Official source

https://en.wikipedia.org/wiki/Vector_quantization

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related courses (8)

MICRO-511: Image processing I

Introduction to the basic techniques of image processing. Introduction to the development of image-processing software and to prototyping using Jupyter notebooks. Application to real-world examples in industrial vision and biomedical imaging. ...

COM-506: Seminar on applied cryptography

This seminar introduces the participants to the current trends, problems, and methods in the area of communication security. ...

CS-401: Applied data analysis

This course teaches the basic techniques, methodologies, and practical skills required to draw meaningful insights from a variety of data, with the help of the most acclaimed software tools in the data science world (pandas, scikit-learn, Spark, etc.) ...

Related lectures (27)

Model Compression: Techniques for Efficient NLP Models

Explores model compression techniques in NLP, discussing pruning, quantization, weight factorization, knowledge distillation, and attention mechanisms.

Neural Signal Compression

Explores analog-to-digital conversion, neural signal optimization, multichannel architectures, and on-chip compression techniques in neuroengineering.

Model Compression Techniques: Enhancing Neural Networks

Covers model compression techniques to enhance the efficiency of large language models in production settings.

Related publications (15)

Compressing DNNs Using Microscaling Formats with Sensitivity and Sparsity

Widespread adoption of deep neural networks (DNNs), coupled with the trillion-parameter scale of modern models, has pushed compute, memory, and power demands of training and inference to their limits, making DNN compression optimization essential for continued AI progress. Microscaling numerical formats have recently emerged as a powerful approach for low-precision training and inference, using block-wise representations that share an exponent across small groups of low-bitwidth elements. This thesis studie

EPFL2026

Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models

Large Language Models (LLMs) are powerful tools for modern applications, but their computational demands limit accessibility. Quantization offers efficiency gains, yet its impact on safety and trustworthiness remains poorly understood. To address this, we introduce OpenMiniSafety, a human-curated safety dataset with 1.067 challenging questions to rigorously evaluate model behavior. We publicly release human safety evaluations for four LLMs (both quantized and full-precision), totaling 4.268 annotated questi

Institute of Electrical and Electronics Engineers (IEEE)2026

Group-and-cut approach for dynamic programming with Fréchet-distributed quantizers

Multi-stage stochastic optimization is a well-known quantitative tool applied in a wide variety of decision-making problems. In this article, we focus on generalized flood risk management problems with Fréchet distributions used to describe the uncertainty. Theoretical solutions of such problems can be found explicitly only in exceptional cases due to their variational form and interdependency of uncertainty in time, e.g., due to cascading impacts of extreme floods. Nevertheless, numerical methods based on

2026

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Official source

https://en.wikipedia.org/wiki/Vector_quantization

About this result

Related courses (8)

MICRO-511: Image processing I

COM-506: Seminar on applied cryptography

This seminar introduces the participants to the current trends, problems, and methods in the area of communication security. ...

CS-401: Applied data analysis

Related lectures (27)

Model Compression: Techniques for Efficient NLP Models

Explores model compression techniques in NLP, discussing pruning, quantization, weight factorization, knowledge distillation, and attention mechanisms.

Neural Signal Compression

Explores analog-to-digital conversion, neural signal optimization, multichannel architectures, and on-chip compression techniques in neuroengineering.

Model Compression Techniques: Enhancing Neural Networks

Covers model compression techniques to enhance the efficiency of large language models in production settings.

Related publications (15)

Compressing DNNs Using Microscaling Formats with Sensitivity and Sparsity

EPFL2026

Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models

Institute of Electrical and Electronics Engineers (IEEE)2026

Group-and-cut approach for dynamic programming with Fréchet-distributed quantizers

2026

Related people (27)

Touradj Ebrahimi

Touradj EBRAHIMI received his M.Sc. and Ph.D., both in Electrical Engineering, from the Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland, in 1989 and 1992 respectively. In 1993, he was a research engineer at the Corporate Research Laboratories of Sony Corporation in Tokyo, where he conducted research on advanced video compression techniques for storage applications. In 1994, he served as a research consultant at AT&T Bell Laboratories working on very low bitrate video coding. He is currently Professor at EPFL heading its Multimedia Signal Processing Group. He is also the Convenor of JPEG standardization Committee. He was also adjunct Professor with the Center of Quantifiable Quality of Service at Norwegian University of Science and Technology (NTNU)from 2008 to 2012. Prof. Ebrahimi has been the recipient of various distinctions and awards, such as the IEEE and Swiss national ASE award, the SNF-PROFILE grant for advanced researchers, Four ISO-Certificates for key contributions to MPEG-4 and JPEG 2000, and the best paper award of IEEE Trans. on Consumer Electronics . He became a Fellow of the international society for optical engineering (SPIE) in 2003. Prof. Ebrahimi has initiated more than two dozen National, European and International cooperation projects with leading companies and research institutes around the world. He is a co-founder of Genista SA, a high-tech start-up company in the field of multimedia quality metrics. In 2002, he founded Emitall SA, start-up active in the area of media security and surveillance. In 2005, he founded EMITALL Surveillance SA, a start-up active in the field of privacy and protection. He is or has been associate Editor with various IEEE, SPIE, and EURASIP journals, such as IEEE Signal Processing Magazine, IEEE Trans. on Image Processing, IEEE Trans. on Multimedia, EURASIP Image Communication Journal, EURASIP Journal of Applied Signal Processing, SPIE Optical Engineering Magazine. Prof. Ebrahimi is a member of Scientific Advisory Board of various start-up and established companies in the general field of Information Technology. He has served as Scientific Expert and Evaluator for Research Funding Agencies such as those of European Commission, The Greek Ministry of Development, The Austrian National Foundation for Scientific Research, The Portuguese Science Foundation, as well as a number of Venture Capital Companies active in the field of Information Technologies and Communication Systems. His research interests include still, moving, and 3D image processing and coding, visual information security (rights protection, watermarking, authentication, data integrity, steganography), new media, and human computer interfaces (smart vision, brain computer interface). He is the author or the co-author of more than 200 research publications, and holds 14 patents. Prof. Ebrahimi is a member of IEEE, SPIE, ACM and IS&T. See the URL below for more details: http://mmspl.epfl.ch

Olivier Verscheure

SWISS DATA SCIENCE CENTER Housed at both EPFL and ETH Zurich, the newly created Swiss Data Science Center (SDSC) will address the fragmentation within today's data and analytics landscapes. The SDSC will be composed of a distributed multi-disciplinary team of data scientists and experts in domains including personalized health and personalized medicine, transportation, earth and environmental science, social science and digital humanities, and economics. The center aims to federate data providers, data and

Pierre Vandergheynst

Pierre Vandergheynst received the M.S. degree in physics and the Ph.D. degree in mathematical physics from the Université catholique de Louvain, Louvain-la-Neuve, Belgium, in 1995 and 1998, respectively. From 1998 to 2001, he was a Postdoctoral Researcher with the Signal Processing Laboratory, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland. He was Assistant Professor at EPFL (2002-2007), where he is now a Full Professor of Electrical Engineering and, by courtesy, of Computer and Communication Sciences. As of 2015, Prof. Vandergheynst serves as EPFL’s Vice-Provost for Education. His research focuses on harmonic analysis, sparse approximations and mathematical data processing in general with applications covering signal, image and high dimensional data processing, computer vision, machine learning, data science and graph-based data processing. He was co-Editor-in-Chief of Signal Processing (2002-2006), Associate Editor of the IEEE Transactions on Signal Processing (2007-2011), the flagship journal of the signal processing community and currently serves as Associate Editor of Computer Vision and Image Understanding and SIAM Imaging Sciences. He has been on the Technical Committee of various conferences, serves on the steering committee of the SPARS workshop and was co-General Chairman of the EUSIPCO 2008 conference. Pierre Vandergheynst is the author or co-author of more than 70 journal papers, one monograph and several book chapters. He has received two IEEE best paper awards. Professor Vandergheynst is a laureate of the Apple 2007 ARTS award and of the 2009-2010 De Boelpaepe prize of the Royal Academy of Sciences of Belgium.

Mathew Magimai Doss

Teaching & PhD PhD Students Karl El Hajal, Tilak Purohit Past EPFL PhD Students as codirector Ramya Rasipuram, Marzieh Razavi, Hannah Muckenhirn, Sandrine Tornay, Subrahmanya Pavankumar Dubagunta, Enno Hermann, Julian David Fritsch, Neha Tarigopula, Mostaani Zohreh, Eklavya Sarkar Courses Automatic speech processing EE-554 The goal of this course is to provide the students with the main formalisms, models and algorithms required for the implementation of advanced speech processing applicat

Babak Falsafi

Babak is a Professor in the School of Computer and Communication Sciences and the founding director of the EcoCloud, an industrial/academic consortium at EPFL investigating scalable data-centric technologies. He has made numerous contributions to computer system design and evaluation including a scalable multiprocessor architecture which was prototyped by Sun Microsystems (now Oracle), snoop filters and memory streaming technologies that are incorporated into IBM BlueGene/P and Q and ARM cores, and computer system performance evaluation methodologies that have been in use by AMD, HP and Google PerKit . He has shown that hardware memory consistency models are neither necessary (in the 90's) nor sufficient (a decade later) to achieve high performance in multiprocessor systems. These results eventually led to fence speculation in modern microprocessors. His latest work on workload-optimized server processors laid the foundation for the first generation of Cavium ARM server CPUs, ThunderX. He is a recipient of an NSF CAREER award, IBM Faculty Partnership Awards, and an Alfred P. Sloan Research Fellowship. He is a fellow of IEEE and ACM.

João Luís Prado Vieira

Mathieu Salzmann

I am a Senior Researcher at EPFL-CVLab, and, since May 2020, an Artificial Intelligence Engineer at ClearSpace (50%). Previously, I was a Senior Researcher and Research Leader in NICTA's computer vision research group. Prior to this, from Sept. 2010 to Jan 2012, I was a Research Assistant Professor at TTI-Chicago, and, from Feb. 2009 to Aug. 2010, a postdoctoral fellow at ICSI and EECS at UC Berkeley under the supervision of Prof. Trevor Darrell. I obtained my PhD in Jan. 2009 from EPFL under the supervision of Prof. Pascal Fua.

Pascal Frossard

Teaching & PhD PhD Students Yiming Qin, Alba Carballo Castro, Elisa Messori, Ke Wang, Sevda Ögüt, Tuna Alikasifoglu, William Cappelletti, Manuel Madeira, Mahdi Amiri, Ortal Yona Senouf, Vasiliki Rizou, Cem Bilaloglu, Abdellah Rahmani, Amel Abdelraheem, Jérémy Jean Philippe Baffou, Adam Hazimeh, Vincent Jung, Yike Zhao Past EPFL PhD Students Dan Jurca, Jean-Paul Wagner, Ivana Radulovic, Ivana Tosic, Effrosyni Kokiopoulou, Zafer Arican, Vijayaraghavan Thirumalai, Tamara Tosic, Eirina Bourtsoulatze, Elif Vur

Francesca De Simone

I am a Post-doctoral fellow in the Signal Processing Laboratory (LTS4) EPFL, Lausanne, Switzerland, since November 2015. From January 2015 to October 2015 I worked as Senior Security Engineer, anti-piracy video streaming expert at Kudelski Group, Cheseaux-sur-Lausanne, Switzerland. Before that, from November 2012 to December 2014 I was Post-doctoral fellow in the Multimedia Signal Processing Lab at Institut Mines Telecom ParisTech, Paris, France, working with Dr. Frederic Dufaux. I received the Ph.D. degree in computer and information science from Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, in 2012 and M.S. degree in electrical engineering from University of Roma Tre, Rome, Italy, in 2006. My research interests concern omnidirectional imaging, multimedia quality assessment, image and video coding, video streaming, human attention modeling, highy dynamic range imaging, image and video representation and analysis, psychophysics, statistics. You can find an updated list of my publications at Google Scholar.

Martin Jaggi

Martin Jaggi is a Tenure Track Assistant Professor at EPFL, heading the Machine Learning and Optimization Laboratory. Before that, he was a post-doctoral researcher at ETH Zurich, at the Simons Institute in Berkeley, and at École Polytechnique in Paris. He has earned his PhD in Machine Learning and Optimization from ETH Zurich in 2011, and a MSc in Mathematics also from ETH Zurich.

Related units (6)

Audiovisual Communications Laboratory

Ebrahimi Group

Signal Processing Laboratory 4

Related concepts (8)

Self-organizing map

A self-organizing map (SOM) or self-organizing feature map (SOFM) is an unsupervised machine learning technique used to produce a low-dimensional (typically two-dimensional) representation of a higher dimensional data set while preserving the topological structure of the data. For example, a data set with variables measured in observations could be represented as clusters of observations with similar values for the variables.

K-means clustering

k-means clustering is a method of vector quantization, originally from signal processing, that aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean (cluster centers or cluster centroid), serving as a prototype of the cluster. This results in a partitioning of the data space into Voronoi cells. k-means clustering minimizes within-cluster variances (squared Euclidean distances), but not regular Euclidean distances, which would be the more difficult Weber problem: the mean optimizes squared errors, whereas only the geometric median minimizes Euclidean distances.

Lloyd's algorithm

In electrical engineering and computer science, Lloyd's algorithm, also known as Voronoi iteration or relaxation, is an algorithm named after Stuart P. Lloyd for finding evenly spaced sets of points in subsets of Euclidean spaces and partitions of these subsets into well-shaped and uniformly sized convex cells. Like the closely related k-means clustering algorithm, it repeatedly finds the centroid of each set in the partition and then re-partitions the input according to which of these centroids is closest.

Related notebooks (6)

Advantages of Digital Transmission over Analog Transmission

[1]Paolo Prandoni, [2]EPFL Analog vs Digital Transmission In this notebook we will explore the advantages of digital transmission over analog transmission. We will model the case of transmission over a long (e.g. transoceanic) cable in which several repeaters are used to compensate for the attenuation incurred by the signal. In [1]: import matplotlib import matplotlib.pyplot as plt import numpy as np import IPython from scipy.io import wavfile plt.rcParams["figure.figsize"] = (14,4)

Exploring Quantization and Oversampling through One-Bit Music

One-Bit Music[1]¶ In this notebook we will talk about quantization and oversampling and we will do so by taking a trip down memory lane to revisit the early from monophonic square waves, introduce polyphony by way of pulse-width modulation and finish with the basics of sigma-delta quantization. pacman In [1]: import matplotlib import matplotlib.pyplot as plt import numpy as np import scipy.signal as sp from scipy.io import wavfile import IPython In [2]: import sys ! wget -q -O mus

Investigating the Effect of Phase on Sound Perception

Hearing the phase of a sound[1]¶

                                      [2]Paolo Prandoni, [3]LCAV, EPFL

In this notebook we will investigate the effect of phase on the perceptual quality of a sound. It is often said that the human ear is commercial-grade audio equipment takes place in the magnitude domain only. In [1]: %matplotlib inline import matplotlib import matplotlib.pyplot as plt import numpy as np import IPython from scipy.io import wavfile In [2]: plt.rcParams["figure.fig

Illustrating Band Structures in Solids: Free Electrons and Atomic Orbitals

In [1]: import numpy as np from numpy import linalg as LA import matplotlib.pyplot as plt

from future import print_function

from ipywidgets import interact, interactive, fixed, interact_manual import ipywidgets as widgets %matplotlib widget The emergence of band structures[1]¶ If you have questions about this notebook, e-mail Oleg Malanyuk (oleg.malanyuk@epfl.ch) This jupyter notebook is intended to provide a illustration of the band structures in solides, starting from the points of v

Understanding and Plotting the Discrete Fourier Transform (DFT)

Plotting the DFT[1]¶

                                      [2]Paolo Prandoni, [3]LCAV, EPFL

In this notebook we will look at the practical issues associated to plotting the DFT and in particular the DFT of real-world signals. We will examine how to map the DFT coefficients to real-world frequencies and we will investigate the frequency resolution of the DFT and the effects of zero padding. In [1]:

first our usual bookkeeping

%matplotlib inline import matplotlib import matplotlib.p