Linear prediction

Linear prediction is a mathematical operation where future values of a discrete-time signal are estimated as a linear function of previous samples. In digital signal processing, linear prediction is often called linear predictive coding (LPC) and can thus be viewed as a subset of filter theory. In system analysis, a subfield of mathematics, linear prediction can be viewed as a part of mathematical modelling or optimization. The most common representation is where is the predicted signal value, the previous observed values, with , and the predictor coefficients. The error generated by this estimate is where is the true signal value. These equations are valid for all types of (one-dimensional) linear prediction. The differences are found in the way the predictor coefficients are chosen. For multi-dimensional signals the error metric is often defined as where is a suitable chosen vector norm. Predictions such as are routinely used within Kalman filters and smoothers to estimate current and past signal values, respectively, from noisy measurements. The most common choice in optimization of parameters is the root mean square criterion which is also called the autocorrelation criterion. In this method we minimize the expected value of the squared error , which yields the equation for 1 ≤ j ≤ p, where R is the autocorrelation of signal xn, defined as and E is the expected value. In the multi-dimensional case this corresponds to minimizing the L2 norm. The above equations are called the normal equations or Yule-Walker equations. In matrix form the equations can be equivalently written as where the autocorrelation matrix is a symmetric, Toeplitz matrix with elements , the vector is the autocorrelation vector , and , the parameter vector. Another, more general, approach is to minimize the sum of squares of the errors defined in the form where the optimisation problem searching over all must now be constrained with .