Interaction information

The interaction information is a generalization of the mutual information for more than two variables. There are many names for interaction information, including amount of information, information correlation, co-information, and simply mutual information. Interaction information expresses the amount of information (redundancy or synergy) bound up in a set of variables, beyond that which is present in any subset of those variables. Unlike the mutual information, the interaction information can be either positive or negative. These functions, their negativity and minima have a direct interpretation in algebraic topology. The conditional mutual information can be used to inductively define the interaction information for any finite number of variables as follows: where Some authors define the interaction information differently, by swapping the two terms being subtracted in the preceding equation. This has the effect of reversing the sign for an odd number of variables. For three variables , the interaction information is given by where is the mutual information between variables and , and is the conditional mutual information between variables and given . The interaction information is symmetric, so it does not matter which variable is conditioned on. This is easy to see when the interaction information is written in terms of entropy and joint entropy, as follows: In general, for the set of variables , the interaction information can be written in the following form (compare with Kirkwood approximation): For three variables, the interaction information measures the influence of a variable on the amount of information shared between and . Because the term can be larger than , the interaction information can be negative as well as positive. This will happen, for example, when and are independent but not conditionally independent given . Positive interaction information indicates that variable inhibits (i.e., accounts for or explains some of) the correlation between and , whereas negative interaction information indicates that variable facilitates or enhances the correlation.