Deterministic annealing for clustering, compression, classification, regression, and related optimization problems

The deterministic annealing approach to clustering and its extensions has demonstrated substantial performance improvement over standard supervised and unsupervised learning methods in a variety of important applications including compression, estimation, pattern recognition and classification, and...

Full description

Saved in:
Bibliographic Details
Published inProceedings of the IEEE Vol. 86; no. 11; pp. 2210 - 2239
Main Author Rose, K.
Format Journal Article
LanguageEnglish
Published IEEE 01.11.1998
Subjects
Annealing
Constraint optimization
Costs
Decision trees
Multilayer perceptrons
Pattern recognition
Physics
Rate-distortion
Temperature
Unsupervised learning
Online AccessGet full text
ISSN0018-9219
DOI10.1109/5.726788

Cover

More Information
Summary:The deterministic annealing approach to clustering and its extensions has demonstrated substantial performance improvement over standard supervised and unsupervised learning methods in a variety of important applications including compression, estimation, pattern recognition and classification, and statistical regression. The application-specific cost is minimized subject to a constraint on the randomness of the solution, which is gradually lowered. We emphasize the intuition gained from analogy to statistical physics. Alternatively the method is derived within rate-distortion theory, where the annealing process is equivalent to computation of Shannon's rate-distortion function, and the annealing temperature is inversely proportional to the slope of the curve. The basic algorithm is extended by incorporating structural constraints to allow optimization of numerous popular structures including vector quantizers, decision trees, multilayer perceptrons, radial basis functions, and mixtures of experts.
ISSN:0018-9219
DOI:10.1109/5.726788