Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 29; no. 9; pp. 1546 - 1562
• Main Authors: Yi Ma, Derksen, H., Wei Hong, Wright, J.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.09.2007; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Asymptotic properties; Bioinformatics; Coding; Compressing; Computer science; control theory; systems; Computer Simulation; Data Clustering; Data compression; Data Compression - methods; Data Interpretation, Statistical; Data Segmentation; Distortion; Exact sciences and technology; Gaussian distribution; Image coding; Image segmentation; Lossy Coding; Lossy Compression; Maximum likelihood estimation; Microarray Data Clustering; Models, Statistical; Multivariate Analysis; Multivariate Mixed Data; Normal Distribution; Optimization; Parameter estimation; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Phase distortion; Rate Distortion; Segmentation; Segments; Signal processing algorithms; Studies; Cluster analysis; Parameter estimation; Multivariate mixed data; Image processing; Data compression; Degenerate system; DNA chip; data clustering; Modeling; Phase transitions; Outlier; lossy coding; Classification; System identification; Pattern analysis; Deterministic approach; Bioinformatics; microarray data clustering; Mixed distribution; lossy compression; Rate distortion theory; data segmentation; Image segmentation; rate distortion; Gaussian process; Optimal solution; Asymptotic approximation; Artificial intelligence
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2007.1085

In this paper, based on ideas from lossy data coding and compression, we present a simple but effective technique for segmenting multivariate mixed data that are drawn from a mixture of Gaussian distributions, which are allowed to be almost degenerate. The goal is to find the optimal segmentation that minimizes the overall coding length of the segmented data, subject to a given distortion. By analyzing the coding length/rate of mixed data, we formally establish some strong connections of data segmentation to many fundamental concepts in lossy data compression and rate-distortion theory. We show that a deterministic segmentation is approximately the (asymptotically) optimal solution for compressing mixed data. We propose a very simple and effective algorithm that depends on a single parameter, the allowable distortion. At any given distortion, the algorithm automatically determines the corresponding number and dimension of the groups and does not involve any parameter estimation. Simulation results reveal intriguing phase-transition-like behaviors of the number of segments when changing the level of distortion or the amount of outliers. Finally, we demonstrate how this technique can be readily applied to segment real imagery and bioinformatic data.