Testing Shape Restrictions of Discrete Distributions

• Published in: Theory of computing systems Vol. 62; no. 1; pp. 4 - 62
• Main Authors: Canonne, Clément L., Diakonikolas, Ilias, Gouleakis, Themis, Rubinfeld, Ronitt
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.01.2018; Springer Nature B.V
• Subjects: Algorithms; Computer Science; Lower bounds; Probability distribution; Studies; Theoretical Aspects of Computer Science; Theory of Computation; Upper bounds; Discrete distributions; Distribution testing; Lower bounds; Probability distributions; Algorithms; Property testing
• ISSN: 1432-4350; 1433-0490; 1433-0490
• DOI: 10.1007/s00224-017-9785-6

We study the question of testing structured properties (classes) of discrete distributions. Specifically, given sample access to an arbitrary distribution D over [ n ] and a property P , the goal is to distinguish between D ∈ P and ℓ 1 ( D , P ) > ε . We develop a general algorithm for this question, which applies to a large range of “shape-constrained” properties, including monotone, log-concave, t -modal, piecewise-polynomial, and Poisson Binomial distributions. Moreover, for all cases considered, our algorithm has near-optimal sample complexity with regard to the domain size and is computationally efficient. For most of these classes, we provide the first non-trivial tester in the literature. In addition, we also describe a generic method to prove lower bounds for this problem, and use it to show our upper bounds are nearly tight. Finally, we extend some of our techniques to tolerant testing, deriving nearly–tight upper and lower bounds for the corresponding questions.