Near-Optimal Coresets of Kernel Density Estimates

• Published in: Discrete & computational geometry Vol. 63; no. 4; pp. 867 - 887
• Main Authors: Phillips, Jeff M., Tai, Wai Ming
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.06.2020; Springer Nature B.V
• Subjects: Combinatorics; Computational Mathematics and Numerical Analysis; Density; Kernels; Lower bounds; Machine learning; Mathematics; Mathematics and Statistics; Polynomials; Upper bounds; Coreset; Discrepancy theory; Kernel density estimate
• ISSN: 0179-5376; 1432-0444; 1432-0444
• DOI: 10.1007/s00454-019-00134-6

We construct near-optimal coresets for kernel density estimates for points in R d when the kernel is positive definite. Specifically we provide a polynomial time construction for a coreset of size O ( d / ε · log 1 / ε ) , and we show a near-matching lower bound of size Ω ( min { d / ε , 1 / ε 2 } ) . When d ≥ 1 / ε 2 , it is known that the size of coreset can be O ( 1 / ε 2 ) . The upper bound is a polynomial-in- ( 1 / ε ) improvement when d ∈ [ 3 , 1 / ε 2 ) and the lower bound is the first known lower bound to depend on d for this problem. Moreover, the upper bound restriction that the kernel is positive definite is significant in that it applies to a wide variety of kernels, specifically those most important for machine learning. This includes kernels for information distances and the sinc kernel which can be negative.