Concept

Precision (computer science)

In computer science, the precision of a numerical quantity is a measure of the detail in which the quantity is expressed. This is usually measured in bits, but sometimes in decimal digits. It is related to precision in mathematics, which describes the number of digits that are used to express a value. Some of the standardized precision formats are Half-precision floating-point format Single-precision floating-point format Double-precision floating-point format Quadruple-precision floating-point format Octuple-precision floating-point format Of these, octuple-precision format is rarely used. The single- and double-precision formats are most widely used and supported on nearly all platforms. The use of half-precision format has been increasing especially in the field of machine learning since many machine learning algorithms are inherently error-tolerant. Precision is often the source of rounding errors in computation. The number of bits used to store a number will often cause some loss of accuracy. An example would be to store "sin(0.1)" in IEEE single precision floating point standard. The error is then often magnified as subsequent computations are made using the data (although it can also be reduced).

Official source

https://en.wikipedia.org/wiki/Precision_(computer_science)

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related courses (3)

CS-173: Fundamentals of digital systems

Welcome to the introductory course in digital design and computer architecture. In this course, we will embark on a journey into the world of digital systems, exploring the fundamental principles and concepts that underpin modern computing technology. ...

MATH-251(a): Numerical analysis

This course presents numerical methods to solve mathematical problems such as systems of linear and nonlinear equations, function approximation, integration and differentiation, and differential equations. ...

CS-455: Topics in theoretical computer science

The students gain an in-depth knowledge of several current and emerging areas of theoretical computer science. The course familiarizes them with advanced techniques, and develops an understanding of fundamental questions that underlie some of the key probl ...

Related lectures (7)

Floating Point Representation in Computers

Covers the representation of real numbers in a computer using floating point representation and the dangers of significant digit cancellation.

Floating Point Numbers: Extended Precision

Explores the representation of numbers using extended floating point notation.

Computing leading eigenvalues of the transfer operator

Explores computing the leading eigenvalue of a transfer operator beyond periodic points, focusing on mathematical settings, spectral radius estimation, and the Zaremba Conjecture.

Related publications (6)

Opening session H2 Policies & initiatives

This paper introduces a randomized Householder QR factorization (RHQR). This factorization can be used to obtain a well-conditioned basis of a linear subspace and thus can be employed in a variety of applications. The RHQR factorization of the input matrix 𝑊 is equivalent to the standard Householder QR factorization of the matrix Ψ⁢𝑊, where Ψ is a sketching matrix that can be obtained from any subspace embedding technique. For this reason, the RHQR factorization can also be reconstructed from the Householde

Fundacio de la Comunitat Valenciana Scito2026

Universal Architectures for the Learning of Polyhedral Norms and Convex Regularizers

Michaël Unser

This paper addresses the task of learning convex regularizers to guide the reconstruction of images from limited data. By imposing that the reconstruction be amplitude-equivariant, we narrow down the class of admissible functionals to those that can be expressed as a power of a seminorm. We then show that such functionals can be approximated to arbitrary precision with the help of polyhedral norms. In particular, we identify two dual parameterizations of such systems: (i) a synthesis (or atomic) form with a

Society for Industrial and Applied Mathematics2026

SCOUT-CT: Sound Constant-Time Outcome with Uncertainty Tracking using multi-taint analysis

Side-channel attacks are an important class of security exploits, in which an attacker gains access to confidential data by observing information inadvertently leaked by a system. Writing constant-time code is a common defense against time-based and microarchitectural side-channel attacks.Many approaches have been proposed to automatically verify that a program is constant-time. Sound methods can detect all information leaks but, to efficiently analyze large programs, most of them rely on overapproximation

Institute of Electrical and Electronics Engineers Inc.2026

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Official source

https://en.wikipedia.org/wiki/Precision_(computer_science)

About this result

Related courses (3)

CS-173: Fundamentals of digital systems

MATH-251(a): Numerical analysis

CS-455: Topics in theoretical computer science

Related lectures (7)

Floating Point Representation in Computers

Covers the representation of real numbers in a computer using floating point representation and the dangers of significant digit cancellation.

Floating Point Numbers: Extended Precision

Explores the representation of numbers using extended floating point notation.

Computing leading eigenvalues of the transfer operator

Explores computing the leading eigenvalue of a transfer operator beyond periodic points, focusing on mathematical settings, spectral radius estimation, and the Zaremba Conjecture.

Related publications (6)

Opening session H2 Policies & initiatives

Fundacio de la Comunitat Valenciana Scito2026

Universal Architectures for the Learning of Polyhedral Norms and Convex Regularizers

Michaël Unser

Society for Industrial and Applied Mathematics2026

SCOUT-CT: Sound Constant-Time Outcome with Uncertainty Tracking using multi-taint analysis

Institute of Electrical and Electronics Engineers Inc.2026

Related people (5)

Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

Michaël Unser

Expertise Image Processing, Medical Imaging, Biological Imaging, Wavelets, Splines, Multiresolution Michael Unser is Full Professor with EPFL’s school of Engineering and the academic director of EPFL's Center for Imaging, Lausanne, Switzerland. His primary areas of investigation are signal processing, biomedical imaging and applied functional analysis. He is internationally recognized for his research contributions to sampling theory, wavelets, the use of splines for image processing, stochastic processes,

Volkan Cevher

Volkan Cevher received the B.Sc. (valedictorian) in electrical engineering from Bilkent University in Ankara, Turkey, in 1999 and the Ph.D. in electrical and computer engineering from the Georgia Institute of Technology in Atlanta, GA in 2005. He was a Research Scientist with the University of Maryland, College Park from 2006-2007 and also with Rice University in Houston, TX, from 2008-2009. Currently, he is an Associate Professor at the Swiss Federal Institute of Technology Lausanne and a Faculty Fellow in the Electrical and Computer Engineering Department at Rice University. His research interests include machine learning, signal processing theory, optimization theory and methods, and information theory. Dr. Cevher is an ELLIS fellow and was the recipient of the Google Faculty Research award in 2018, the IEEE Signal Processing Society Best Paper Award in 2016, a Best Paper Award at CAMSAP in 2015, a Best Paper Award at SPARS in 2009, and an ERC CG in 2016 as well as an ERC StG in 2011.

David Atienza Alonso

David Atienza Alonso is an associate professor of EE and director of the Embedded Systems Laboratory (ESL) at EPFL, Switzerland. He received his MSc and PhD degrees in computer science and engineering from UCM, Spain, and IMEC, Belgium, in 2001 and 2005, respectively. His research interests include system-level design methodologies for multi-processor system-on-chip (MPSoC) servers and edge AI architectures. Dr. Atienza has co-authored more than 350 papers, one book, and 12 patents in these previous areas. He has also received several recognitions and award, among them, the ICCAD 10-Year Retrospective Most Influential Paper Award in 2020, Design Automation Conference (DAC) Under-40 Innovators Award in 2018, the IEEE TCCPS Mid-Career Award in 2018, an ERC Consolidator Grant in 2016, the IEEE CEDA Early Career Award in 2013, the ACM SIGDA Outstanding New Faculty Award in 2012, and a Faculty Award from Sun Labs at Oracle in 2011. He has also earned two best paper awards at the VLSI-SoC 2009 and CST-HPCS 2012 conference, and five best paper award nominations at the DAC 2013, DATE 2013, WEHA-HPCS 2010, ICCAD 2006, and DAC 2004 conferences. He serves or has served as associate editor of IEEE Trans. on Computers (TC), IEEE Design & Test of Computers (D&T), IEEE Trans. on CAD (T-CAD), IEEE Transactions on Sustainable Computing (T-SUSC), and Elsevier Integration. He was the Technical Program Chair of DATE 2015 and General Chair of DATE 2017. He served as President of IEEE CEDA in the period 2018-2019 and was GOLD member of the Board of Governors of IEEE CASS from 2010 to 2012. He is a Distinguished Member of ACM and an IEEE Fellow.

Emmanuel Lanti

Teaching & PhD Courses Parallel programming PHYS-743 Learn the concepts, tools and API's that are needed to debug, test, optimize and parallelize a scientific application on a cluster from an existing code or from scratch. Both OpenMP (shared memory) and MPI (distributed memory) paradigms are presented and experimented.

Related concepts (3)

Single-precision floating-point format

Single-precision floating-point format (sometimes called FP32 or float32) is a computer number format, usually occupying 32 bits in computer memory; it represents a wide dynamic range of numeric values by using a floating radix point. A floating-point variable can represent a wider range of numbers than a fixed-point variable of the same bit width at the cost of precision. A signed 32-bit integer variable has a maximum value of 231 − 1 = 2,147,483,647, whereas an IEEE 754 32-bit base-2 floating-point variable has a maximum value of (2 − 2−23) × 2127 ≈ 3.

Arbitrary-precision arithmetic

In computer science, arbitrary-precision arithmetic, also called bignum arithmetic, multiple-precision arithmetic, or sometimes infinite-precision arithmetic, indicates that calculations are performed on numbers whose digits of precision are limited only by the available memory of the host system. This contrasts with the faster fixed-precision arithmetic found in most arithmetic logic unit (ALU) hardware, which typically offers between 8 and 64 bits of precision.

Double-precision floating-point format

Double-precision floating-point format (sometimes called FP64 or float64) is a floating-point number format, usually occupying 64 bits in computer memory; it represents a wide dynamic range of numeric values by using a floating radix point. Floating point is used to represent fractional values, or when a wider range is needed than is provided by fixed point (of the same bit width), even if at the cost of precision. Double precision may be chosen when the range or precision of single precision would be insufficient.

Related people (5)