In information theory, Rényi entropy refers to a class of measures, of entropy that are essentially logarithms of diversity indices.
For special values of its parameter, the notion of Rényy entropy reproduces all of: Shannon entropy, Hartley entropy/max-entropy and min-entropy.
Let be a probability distribution over elements, and let be a non-negative real number not equal to 1:
The Rényi entropy of at order is:
For various (limiting) values of the Rényi entropy reduces to notions of entropy that are known by their own names:
for , Rényi entropy equals the Hartley entropy/max-entropy,
in the limit , Rényi entropy equals the Shannon entropy,
for , Rényi entropy equals the collision entropy,
in the limit , Rényi entropy equals the min-entropy.
The Rényi entropy is an anti-monotone function in the order-parameter :
(e.g. Ram & Sason 16, Fact 1)
In particular, in terms of the above special cases, this means that
Due to:
Textbook account:
See also
Wikipedia, Rényi entropy
Eshed Ram, Igal Sason, On Renyi Entropy Power Inequalities (arXiv:1601.06555)
On holographic Renyi entropy in relation to holographic entanglement entropy and quantum error correcting codes:
Xi Dong, The Gravity Dual of Renyi Entropy, Nature Communications 7, 12472 (2016) (arXiv:1601.06788, doi:10.1038/ncomms12472)
Chris Akers, Pratik Rath, Holographic Renyi entropy from quantum error correction, J. High Energ. Phys. 2019, 52 (2019) (arXiv:1811.05171)
Last revised on May 28, 2021 at 12:14:23. See the history of this page for a list of all contributions to it.