Jeffreys prior

In Bayesian probability, the Jeffreys prior, named after Sir Harold Jeffreys, is a non-informative prior distribution for a parameter space; its density function is proportional to the square root of the determinant of the Fisher information matrix: It has the key feature that it is invariant under a change of coordinates for the parameter vector . That is, the relative probability assigned to a volume of a probability space using a Jeffreys prior will be the same regardless of the parameterization used to define the Jeffreys prior. This makes it of special interest for use with scale parameters. In maximum likelihood estimation of exponential family models, penalty terms based on the Jeffreys prior were shown to reduce asymptotic bias in point estimates. If and are two possible parametrizations of a statistical model, and is a continuously differentiable function of , we say that the prior is "invariant" under a reparametrization if that is, if the priors and are related by the usual change of variables theorem. Since the Fisher information transforms under reparametrization as defining the priors as and gives us the desired "invariance". Analogous to the one-parameter case, let and be two possible parametrizations of a statistical model, with a continuously differentiable function of . We call the prior "invariant" under reparametrization if where is the Jacobian matrix with entries Since the Fisher information matrix transforms under reparametrization as we have that and thus defining the priors as and gives us the desired "invariance". From a practical and mathematical standpoint, a valid reason to use this non-informative prior instead of others, like the ones obtained through a limit in conjugate families of distributions, is that the relative probability of a volume of the probability space is not dependent upon the set of parameter variables that is chosen to describe parameter space. Sometimes the Jeffreys prior cannot be normalized, and is thus an improper prior.