Monoidal categoryIn mathematics, a monoidal category (or tensor category) is a equipped with a bifunctor that is associative up to a natural isomorphism, and an I that is both a left and right identity for ⊗, again up to a natural isomorphism. The associated natural isomorphisms are subject to certain coherence conditions, which ensure that all the relevant s commute. The ordinary tensor product makes vector spaces, abelian groups, R-modules, or R-algebras into monoidal categories. Monoidal categories can be seen as a generalization of these and other examples.
Category theoryCategory theory is a general theory of mathematical structures and their relations that was introduced by Samuel Eilenberg and Saunders Mac Lane in the middle of the 20th century in their foundational work on algebraic topology. Category theory is used in almost all areas of mathematics. In particular, numerous constructions of new mathematical objects from previous ones that appear similarly in several contexts are conveniently expressed and unified in terms of categories.
Mutual informationIn probability theory and information theory, the mutual information (MI) of two random variables is a measure of the mutual dependence between the two variables. More specifically, it quantifies the "amount of information" (in units such as shannons (bits), nats or hartleys) obtained about one random variable by observing the other random variable. The concept of mutual information is intimately linked to that of entropy of a random variable, a fundamental notion in information theory that quantifies the expected "amount of information" held in a random variable.
Category (mathematics)In mathematics, a category (sometimes called an abstract category to distinguish it from a ) is a collection of "objects" that are linked by "arrows". A category has two basic properties: the ability to compose the arrows associatively and the existence of an identity arrow for each object. A simple example is the , whose objects are sets and whose arrows are functions. is a branch of mathematics that seeks to generalize all of mathematics in terms of categories, independent of what their objects and arrows represent.
Category of small categoriesIn mathematics, specifically in , the category of small categories, denoted by Cat, is the whose objects are all and whose morphisms are functors between categories. Cat may actually be regarded as a with natural transformations serving as 2-morphisms. The initial object of Cat is the empty category 0, which is the category of no objects and no morphisms. The terminal object is the terminal category or trivial category 1 with a single object and morphism. The category Cat is itself a , and therefore not an object of itself.
Accessible categoryThe theory of accessible categories is a part of mathematics, specifically of . It attempts to describe categories in terms of the "size" (a cardinal number) of the operations needed to generate their objects. The theory originates in the work of Grothendieck completed by 1969, and Gabriel and Ulmer (1971). It has been further developed in 1989 by Michael Makkai and Robert Paré, with motivation coming from model theory, a branch of mathematical logic. A standard text book by Adámek and Rosický appeared in 1994.
Interaction informationThe interaction information is a generalization of the mutual information for more than two variables. There are many names for interaction information, including amount of information, information correlation, co-information, and simply mutual information. Interaction information expresses the amount of information (redundancy or synergy) bound up in a set of variables, beyond that which is present in any subset of those variables. Unlike the mutual information, the interaction information can be either positive or negative.
Information theoryInformation theory is the mathematical study of the quantification, storage, and communication of information. The field was originally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940s. The field, in applied mathematics, is at the intersection of probability theory, statistics, computer science, statistical mechanics, information engineering, and electrical engineering. A key measure in information theory is entropy.
Conditional mutual informationIn probability theory, particularly information theory, the conditional mutual information is, in its most basic form, the expected value of the mutual information of two random variables given the value of a third. For random variables , , and with support sets , and , we define the conditional mutual information as This may be written in terms of the expectation operator: . Thus is the expected (with respect to ) Kullback–Leibler divergence from the conditional joint distribution to the product of the conditional marginals and .
Monoid (category theory)In , a branch of mathematics, a monoid (or monoid object, or internal monoid, or algebra) (M, μ, η) in a (C, ⊗, I) is an M together with two morphisms μ: M ⊗ M → M called multiplication, η: I → M called unit, such that the pentagon and the unitor diagram commute. In the above notation, 1 is the identity morphism of M, I is the unit element and α, λ and ρ are respectively the associativity, the left identity and the right identity of the monoidal category C. Dually, a comonoid in a monoidal category C is a monoid in the Cop.
Variation of informationIn probability theory and information theory, the variation of information or shared information distance is a measure of the distance between two clusterings (partitions of elements). It is closely related to mutual information; indeed, it is a simple linear expression involving the mutual information. Unlike the mutual information, however, the variation of information is a true metric, in that it obeys the triangle inequality. Suppose we have two partitions and of a set into disjoint subsets, namely and .
Dual objectIn , a branch of mathematics, a dual object is an analogue of a dual vector space from linear algebra for in arbitrary . It is only a partial generalization, based upon the categorical properties of duality for finite-dimensional vector spaces. An object admitting a dual is called a dualizable object. In this formalism, infinite-dimensional vector spaces are not dualizable, since the dual vector space V∗ doesn't satisfy the axioms. Often, an object is dualizable only when it satisfies some finiteness or compactness property.
Convolutional neural networkConvolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.
Principal component analysisPrincipal component analysis (PCA) is a popular technique for analyzing large datasets containing a high number of dimensions/features per observation, increasing the interpretability of data while preserving the maximum amount of information, and enabling the visualization of multidimensional data. Formally, PCA is a statistical technique for reducing the dimensionality of a dataset. This is accomplished by linearly transforming the data into a new coordinate system where (most of) the variation in the data can be described with fewer dimensions than the initial data.
AutoencoderAn autoencoder is a type of artificial neural network used to learn efficient codings of unlabeled data (unsupervised learning). An autoencoder learns two functions: an encoding function that transforms the input data, and a decoding function that recreates the input data from the encoded representation. The autoencoder learns an efficient representation (encoding) for a set of data, typically for dimensionality reduction. Variants exist, aiming to force the learned representations to assume useful properties.
Deep belief networkIn machine learning, a deep belief network (DBN) is a generative graphical model, or alternatively a class of deep neural network, composed of multiple layers of latent variables ("hidden units"), with connections between the layers but not between units within each layer. When trained on a set of examples without supervision, a DBN can learn to probabilistically reconstruct its inputs. The layers then act as feature detectors. After this learning step, a DBN can be further trained with supervision to perform classification.
Digital image processingDigital image processing is the use of a digital computer to process s through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over . It allows a much wider range of algorithms to be applied to the input data and can avoid problems such as the build-up of noise and distortion during processing. Since images are defined over two dimensions (perhaps more) digital image processing may be modeled in the form of multidimensional systems.
Computer visionComputer vision tasks include methods for , , and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images (the input to the retina in the human analog) into descriptions of the world that make sense to thought processes and can elicit appropriate action.
Unsupervised learningUnsupervised learning, is paradigm in machine learning where, in contrast to supervised learning and semi-supervised learning, algorithms learn patterns exclusively from unlabeled data. Neural network tasks are often categorized as discriminative (recognition) or generative (imagination). Often but not always, discriminative tasks use supervised methods and generative tasks use unsupervised (see Venn diagram); however, the separation is very hazy. For example, object recognition favors supervised learning but unsupervised learning can also cluster objects into groups.
Structured programmingStructured programming is a programming paradigm aimed at improving the clarity, quality, and development time of a computer program by making extensive use of the structured control flow constructs of selection (if/then/else) and repetition (while and for), block structures, and subroutines. It emerged in the late 1950s with the appearance of the ALGOL 58 and ALGOL 60 programming languages, with the latter including support for block structures.
