Incremental Clustering and Dynamic Information Retrieval

• Published in: SIAM journal on computing Vol. 33; no. 6; pp. 1417 - 1440
• Main Authors: Charikar, Moses, Chekuri, Chandra, Feder, Tomas, Motwani, Rajeev
• Format: Journal Article
• Language: English
• Published: Philadelphia, PA Society for Industrial and Applied Mathematics 01.01.2004
• Subjects: Algorithmics. Computability. Computer arithmetics; Algorithms; Applied sciences; Clustering; Computer science; Computer science; control theory; systems; Documents; Exact sciences and technology; Information retrieval; Queries; Scholarships & fellowships; Theoretical computing; Lower bound; Metric space; Computer theory; Algorithm performance; Greedy algorithm; K center; Incremental clustering; Information retrieval; Dynamic information; Minimum diameter clustering; Algorithm analysis; Agglomerative clustering
• ISSN: 0097-5397; 1095-7111
• DOI: 10.1137/S0097539702418498

Motivated by applications such as document and image classification in information retrieval, we consider the problem of clustering dynamic point sets in a metric space. We propose a model called incremental clustering which is based on a careful analysis of the requirements of the information retrieval application, and which should also be useful in other applications. The goal is to efficiently maintain clusters of small diameter as new points are inserted. We analyze several natural greedy algorithms and demonstrate that they perform poorly. We propose new deterministic and randomized incremental clustering algorithms which have a provably good performance, and which we believe should also perform well in practice. We complement our positive results with lower bounds on the performance of incremental algorithms. Finally, we consider the dual clustering problem where the clusters are of fixed diameter, and the goal is to minimize the number of clusters.