Model-Based Clustering by Probabilistic Self-Organizing Maps

• Published in: IEEE transactions on neural networks Vol. 20; no. 5; pp. 805 - 826
• Main Authors: CHENG, Shih-Sian, FU, Hsin-Chia, WANG, Hsin-Min
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.05.2009; Institute of Electrical and Electronics Engineers
• Subjects: Algorithms; Annealing; Applied sciences; Artificial intelligence; Classification; Classification algorithms; Classification expectation-maximization (CEM) algorithm; Clustering; Clustering algorithms; Computer science; Computer science; control theory; systems; Connectionism. Neural networks; Data processing. List processing. Character string processing; Data visualization; deterministic annealing expectation-maximization (DAEM) algorithm; Exact sciences and technology; expectation-maximization (EM) algorithm; Gaussian distribution; Information science; Iterative algorithms; Learning; Memory organisation. Data processing; model-based clustering; Neural networks; Probabilistic methods; probabilistic self-organizing map (PbSOM); Probability theory; Self organizing feature maps; self-organizing map (SOM); Software; Cluster analysis; Gaussian distribution; Mixture theory; Modeling; deterministic annealing expectation―maximization (DAEM) algorithm; Self organization; Semantics; Classification; self-organizing map (SOM); Probability learning; Learning algorithm; Deterministic approach; Annealing; Probabilistic approach; Preservation; Multivariate distribution; Neural network; Topology; Kohonen algorithm; model-based clustering; probabilistic self-organizing map (PbSOM); Conceptual analysis; Classification expectation―maximization (CEM) algorithm; Gaussian process; Maximum likelihood; EM algorithm; Artificial intelligence; expectation―maximization (EM) algorithm
• ISSN: 1045-9227; 1941-0093; 1941-0093
• DOI: 10.1109/TNN.2009.2013708

In this paper, we consider the learning process of a probabilistic self-organizing map (PbSOM) as a model-based data clustering procedure that preserves the topological relationships between data clusters in a neural network. Based on this concept, we develop a coupling-likelihood mixture model for the PbSOM that extends the reference vectors in Kohonen's self-organizing map (SOM) to multivariate Gaussian distributions. We also derive three expectation-maximization (EM)-type algorithms, called the SOCEM, SOEM, and SODAEM algorithms, for learning the model (PbSOM) based on the maximum-likelihood criterion. SOCEM is derived by using the classification EM (CEM) algorithm to maximize the classification likelihood; SOEM is derived by using the EM algorithm to maximize the mixture likelihood; and SODAEM is a deterministic annealing (DA) variant of SOCEM and SOEM. Moreover, by shrinking the neighborhood size, SOCEM and SOEM can be interpreted, respectively, as DA variants of the CEM and EM algorithms for Gaussian model-based clustering. The experimental results show that the proposed PbSOM learning algorithms achieve comparable data clustering performance to that of the deterministic annealing EM (DAEM) approach, while maintaining the topology-preserving property.