An Introduction to Kolmogorov Complexity and Its Applications

Written by two experts in the field, this is the only comprehensive and unified treatment of the central ideas and applications of Kolmogorov complexity. The book presents a thorough treatment of the subject with a wide range of illustrative applications.

Saved in:
Bibliographic Details
Main Author Li, Ming
Format eBook Book
LanguageEnglish
Published New York Springer Nature 2008
Springer
Springer New York
Edition1
SeriesTexts in Computer Science
Subjects
Algorithms
Applications of Mathematics
Coding and Information Theory
Computer Science
Electronic data processing
Kolmogorov complexity
Mathematics
Pattern Recognition
Probabilities & applied mathematics
Statistical Theory and Methods
Theory of Computation
Online AccessGet full text
ISBN0387498206
9780387498201
9780387339986
0387339981
ISSN1868-0941
1868-095X
DOI10.1007/978-0-387-49820-1

Cover

Table of Contents:
  • Intro -- Preface to the First Edition -- Preface to the Second Edition -- Preface to the Third Edition -- How to Use This Book -- Outlines of One-Semester Courses -- Contents -- List of Figures -- List of Figures -- 1 Preliminaries -- 2 Algorithmic Complexity -- 3 Algorithmic Prefix Complexity -- 4 Algorithmic Probability -- 5 Inductive Reasoning -- 6 The Incompressibility Method -- 7 Resource-Bounded Complexity -- 8 Physics, Information, and Computation -- References -- Index
  • 6.1 Three Examples -- 6.1.1 Computation Time of Turing Machines -- 6.1.2 Adding Fast-On Average -- 6.1.3 Boolean Matrix Rank -- Exercises -- 6.2 High-Probability Properties -- 6.3 Combinatorics -- 6.3.1 Transitive Tournament -- 6.3.2 Tournament with k-Dominators -- 6.3.3 Ramsey Numbers -- 6.3.4 Coin-Weighing Problem -- 6.3.5 High-Probability Properties Revisited -- Exercises -- 6.4 Kolmogorov Random Graphs -- 6.4.1 Statistics of Subgraphs -- 6.4.2 Unlabeled Graph Counting -- Exercises -- 6.5 Compact Routing -- 6.5.1 Upper Bound -- 6.5.2 Lower Bound -- 6.5.3 Average Case -- Exercises -- 6.6 Average-Case Analysis of Sorting -- 6.6.1 Heapsort -- 6.6.2 Shellsort -- Exercises -- 6.7 Longest Common Subsequence -- Exercises -- 6.8 Formal Language Theory -- Exercises -- 6.9 Online CFL Recognition -- Exercises -- 6.10 Turing Machine Time Complexity -- Exercises -- 6.11 Communication Complexity -- 6.11.1 Identity Function -- 6.11.2 Inner Product Function -- Exercises -- 6.12 Circuit Complexity -- 6.13 History and References -- Chapter 7 -- Resource-Bounded Complexity -- 7.1 Mathematical Theory -- 7.1.1 Computable Majorants -- 7.1.2 Resource-Bounded Hierarchies -- Exercises -- 7.2 Language Compression -- 7.2.1 Decision Compression -- 7.2.2 Description Compression -- 7.2.3Ranking -- Exercises -- 7.3 Computa-tional Complexity -- 7.3.1 Constructing Oracles -- 7.3.2 P-Printability -- 7.3.3 Derandomization -- Exercises -- 7.4 Instance Complexity -- Exercises -- 7.5 Kt and Universal Search -- 7.5.1 Universal Optimal Search -- Exercises -- 7.6 Time-Limited Universal Distributions -- Exercises -- 7.7 Logical Depth -- Exercises -- 7.8 History and References -- Chapter 8 -- Physics, Information, and Computation -- 8.1 Information Theory -- 8.1.1 Algorithmic Complexity and Shannon's Entropy -- 8.1.2 Mutual Information -- 8.1.3 Mutual Information Nonincrease
  • Intro -- Text in Computer Science -- Series Title Page -- Title Page -- Copyright Page -- Preface to the First Edition -- Preface to the Second Edition -- Preface to the Third Edition -- How to Use This Book -- Outlines of One-Semester Courses -- Contents -- List of Figures -- Chapter 1 -- Preliminaries -- 1.1 A Brief Introduction -- 1.2 Prerequisites and Notation -- 1.3 Numbers and Combinatorics -- Exercises -- 1.4 Binary Strings -- Exercises -- 1.5 Asymptotic Notation -- Exercises -- 1.6 Basics of Probability Theory -- 1.6.1 Kolmogorov Axioms -- 1.6.2 Conditional Probability -- 1.6.3 Continuous Sample Spaces -- Exercises -- 1.7 Basics of Computability Theory -- 1.7.1 Effective Enumerations and Universal Machines -- 1.7.2 Undecidability of the Halting Problem -- 1.7.3 Semi-Computable Functions -- 1.7.4 Feasible Computations -- Exercises -- 1.8 The Roots of Kolmogorov Complexity -- 1.8.1 A Lacuna of Classical Probability Theory -- 1.8.2 A Lacuna of Information Theory -- 1.9 Randomness -- Exercises -- 1.10 Prediction and Probability -- Exercises -- 1.11 Information Theory and Coding -- 1.11.1 Prefix-Codes -- 1.11.2 The Kraft Inequality -- 1.11.3 Optimal Codes -- 1.11.4 Universal Codes -- 1.11.5 Statistics -- 1.11.6 Rate Distortion -- Exercises -- 1.12 State × Symbol Complexity -- Exercises -- 1.13 History and References -- Chapter 2 -- Algorithmic Complexity -- 2.1 The Invariance Theorem -- 2.1.1 Two-Part Codes -- 2.1.2 Upper Bounds -- 2.1.3 Invariance of Kolmogorov Complexity -- 2.1.4 Concrete Kolmogorov Complexity -- Exercises -- 2.2 Incompress-ibility -- 2.2.1 Randomness Deficiency -- Exercises -- 2.3 C as an Integer Function -- Exercises -- 2.4 Random Finite Sequences -- 2.4.1 Randomness Tests -- 2.4.2 Explicit Universal Randomness Test -- Exercises -- 2.5 *Random Infinite Sequences -- 2.5.1 Complexity Oscillations
  • 4.4 Universal Average-Case Complexity -- Exercises -- 4.5 Continuous Sample Space -- 4.5.1 Universal Enumerable Semimeasure -- 4.5.2 A Priori Probability -- 4.5.3 *Solomonoff Normalization -- 4.5.4 *Monotone Complexity and a Coding Theorem -- 4.5.5 *Relation Between Complexities -- 4.5.6 *Randomness by Integral Tests -- 4.5.7 *Randomness by Martingale Tests -- 4.5.8 *Randomness by Martingales -- 4.5.9 *Relations Between Tests -- Exercises -- 4.6 Universal Average-Case Complexity, Continued -- 4.7 History and References -- Chapter 5 -- Inductive Reasoning -- 5.1 Introduction -- 5.1.1 Epicurus's Principle -- 5.1.2 Occam's Razor -- 5.1.3 Bayes's Rule -- 5.1.4 Hume on Induction -- 5.1.5 Hypothesis Identification and Prediction by Compression -- 5.2 Solomonoff's Theory of Prediction -- 5.2.1 Universal Prediction -- Convergence in Difference -- Convergence in Ratio -- 5.2.4 Prediction by Data Compression -- 5.2.5 Universal Recursion Induction -- Hypothesis Identification -- Mistake Bounds -- Certification -- Exercises -- 5.3 Simple Pac-Learning -- 5.3.1 Pac-Learning -- 5.3.2 Occam's Razor Formalized -- 5.3.3 Making Pac-Learning Simple -- 5.3.4 Discrete Sample Space -- 5.3.5 Continuous Sample Space -- Exercises -- 5.4 Hypothesis Identification by MDL -- 5.4.1 Ideal MDL -- 5.4.2 Logarithmic Version of Bayes's Rule -- 5.4.3 Discrepancy Between Probability and Complexity -- 5.4.4 Resolving the Discrepancy -- 5.4.5 Applying Minimum Description Length -- Exercises -- 5.5 Nonprobabilistic Statistics -- 5.5.1 Algorithmic Sufficient Statistic -- 5.5.2 Structure Functions -- Best-Fit Estimator -- Maximum-Likelihood Estimator -- MDL Estimator -- Relations Between the Structure Functions -- Shapes -- 5.5.8 Foundations of MDL -- 5.5.9 Explanation and Interpretation -- Exercises -- 5.6 History and References -- Chapter 6 -- The Incompressibility Method
  • 8.1.4 Rate Distortion -- Distortion Measures -- Every Shape -- Fitness of Destination Word -- Characterization -- Algorithmic versus Probabilistic Rate Distortion -- Exercises -- 8.2 Reversible Computation -- 8.2.1 Energy Dissipation -- 8.2.2 Reversible Logic Circuits -- 8.2.3 Reversible Ballistic Computer -- 8.2.4 Thermodynamics of Computing -- 8.2.5 Reversible Turing Machines -- Exercises -- 8.3 Information Distance -- 8.3.1 Definitions -- 8.3.2 Maximal Overlap -- 8.3.3 Universality -- 8.3.4 Reversible Distance -- 8.3.5 Sum Distance -- 8.3.6 Metrics Relations -- 8.3.7 Minimal Overlap -- Exercises -- 8.4 Normalized Information Distance -- 8.4.1 The Similarity Metric -- 8.4.2 Applications of Normalized Information Distance -- Unification -- Exercises -- 8.5 Thermo-dynamics -- 8.5.1 Classical Entropy -- 8.5.2 Statistical Mechanics and Boltzmann Entropy -- 8.5.3 Gibbs Entropy -- 8.6 Entropy Revisited -- 8.6.1 Algorithmic Entropy -- 8.6.2 Algorithmic Entropy and Randomness Tests -- Exercises -- 8.7 Quantum Kolmogorov Complexity -- 8.7.1 Quantum Computation -- 8.7.2 Quantum Turing Machine -- 8.7.3 Classical Descriptions of Pure Quantum States -- 8.7.4 Properties -- Exercises -- 8.8 Compressionin Nature -- 8.8.1 Compression by Ants -- 8.8.2 Compression by Science -- 8.9 History and References -- References -- Index -- Text in Computer Science
  • 2.5.2 Sequential Randomness Tests -- 2.5.3 Characterization of Random Sequences -- Exercises -- 2.6 Statistical Properties of Finite Sequences -- 2.6.1 Statistics of 0's and 1's -- 2.6.2 Statistics of Blocks -- 2.6.3 Length of Runs -- Exercises -- 2.7 Algorithmic Properties of C -- 2.7.1 Undecidability by Incompressibility -- 2.7.2 Barzdins's Lemma -- Exercises -- 2.8 Algorithmic Information Theory -- 2.8.1 Entropy, Information, and Complexity -- 2.8.2 Symmetry of Information -- Exercises -- 2.9 History and References -- Chapter 3 -- Algorithmic Prefix Complexity -- 3.1 The Invariance Theorem -- Exercises -- 3.2 *Sizes of the Constants -- 3.2.1 Encoding Combinators as Binary Strings -- 3.2.2 Encoding Booleans, Pairs, and Binary Strings as Combinators -- 3.2.3 Output Conventions -- 3.2.4 The Universal Combinator -- 3.2.5 Decoding a Combinator Encoding -- 3.2.6 Concrete Complexities -- 3.3 Incompress-ibility -- Exercises -- 3.4 K as anInteger Function -- Exercises -- 3.5 Random Finite Sequences -- Exercises -- 3.6 *RandomInfinite Sequences -- 3.6.1 Explicit Universal Randomness Tests -- 3.6.2 Halting Probability -- Exercises -- 3.7 Algorithmic Properties of K -- 3.7.1 Randomness in Diophantine Equations -- Exercises -- 3.8 *Complexity of Complexity -- 3.9 *Symmetry of Algorithmic Information -- 3.9.1 Algorithmic Information and Entropy -- 3.9.2 Exact Symmetry of Information -- Exercises -- 3.10 History and References -- Chapter 4 -- Algorithmic Probability -- 4.1 Semicomput-able Functions Revisited -- 4.2 Measure Theory -- Exercises -- 4.3 Discrete Sample Space -- 4.3.1 Universal Lower Semicomputable Semimeasure -- 4.3.2 A Priori Probability -- 4.3.3 Algorithmic Probability -- 4.3.4 The Coding Theorem -- 4.3.5 Randomness by Sum Tests -- 4.3.6 Randomness by Universal Gambling -- Betting Against a Crooked Player -- Exercises