Greedy Algorithms for Optimal Distribution Approximation

The approximation of a discrete probability distribution \(\mathbf{t}\) by an \(M\)-type distribution \(\mathbf{p}\) is considered. The approximation error is measured by the informational divergence \(\mathbb{D}(\mathbf{t}\Vert\mathbf{p})\), which is an appropriate measure, e.g., in the context of...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Geiger, Bernhard C, Böcherer, Georg
Format Paper Journal Article
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 22.01.2016
Subjects
Approximation
Compression tests
Computer Science - Information Theory
Data compression
Divergence
Error analysis
Greedy algorithms
Mathematical analysis
Mathematics - Information Theory
Online AccessGet full text
ISSN2331-8422
DOI10.48550/arxiv.1601.06039

Cover

More Information
Summary:The approximation of a discrete probability distribution \(\mathbf{t}\) by an \(M\)-type distribution \(\mathbf{p}\) is considered. The approximation error is measured by the informational divergence \(\mathbb{D}(\mathbf{t}\Vert\mathbf{p})\), which is an appropriate measure, e.g., in the context of data compression. Properties of the optimal approximation are derived and bounds on the approximation error are presented, which are asymptotically tight. It is shown that \(M\)-type approximations that minimize either \(\mathbb{D}(\mathbf{t}\Vert\mathbf{p})\), or \(\mathbb{D}(\mathbf{p}\Vert\mathbf{t})\), or the variational distance \(\Vert\mathbf{p}-\mathbf{t}\Vert_1\) can all be found by using specific instances of the same general greedy algorithm.
Bibliography:SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
ISSN:2331-8422
DOI:10.48550/arxiv.1601.06039