The notion of relative entropy of states is a generalization of the notion of entropy to a situation where the entropy of one state is measured “relative to” another state.
is also called
Kullback-Leibler divergence
information divergence
information gain .
For two finite probability distributions and , their relative entropy is
Alternatively, for two density matrices, their relative entropy is
For a measurable space and and two probability measures on , such that is absolutely continuous with respect to , their relative entropy is the integral
where is the Radon-Nikodym derivative of with respect to .
Let be a von Neumann algebra and let , be two states on it (faithful, positive linear functionals).
The relative entropy of relative to is
where is the relative modular operator? of any cyclic and separating vector representatives and of and .
This is due to (Araki).
This definition is independent of the choice of these representatives.
In the case that is finite dimensional and and are density matrices of and , respectively, this reduces to the above definition.
Relative entropy of states on von Neumann algebras was introduced in
A characterization of relative entropy on finite-dimensional C-star algebras is given in
A survey of entropy in operator algebras is in
Last revised on July 15, 2011 at 22:12:00. See the history of this page for a list of all contributions to it.