Towards a Methodology to Search for Near-Optimal Representations in Classification Problems

• Published in: Artificial Intelligence and Knowledge Engineering Applications: A Bioinspired Approach pp. 291 - 299
• Main Authors: del Valle, Manuel, Sánchez, Beatriz, Lago-Fernández, Luis F., Corbacho, Fernando J.
• Format: Book Chapter Conference Proceeding
• Language: English
• Published: Berlin, Heidelberg Springer Berlin Heidelberg 2005; Springer
• Series: Lecture Notes in Computer Science
• Subjects: Applied sciences; Artificial intelligence; Computer science; control theory; systems; Exact sciences and technology; Fisher Linear Discriminant; General Genetic Algorithm; Genetic Algorithm; Supervise Learning Algorithm; Trigonometric Basis; Knowledge engineering; Gradient; Statistical analysis; Probabilistic approach; Periodicity; Cognition; Decision tree; Remote teaching; Genetic algorithm; Classification; Multilayer perceptrons; Objective function; Internet; Artificial intelligence
• ISBN: 9783540263197; 3540263195
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/11499305_30

This paper provides a first step towards a methodology that allows the search for near-optimal representations in classification problems by combining feature transformations from an initial family of basis functions. The original representation for the problem data may not be the most appropriate, and therefore it might be necessary to search for a new representation space that is closer to the structure of the problem to be solved. The outcome of this search is critical for the successful solution of the problem. For instance, if the objective function has certain global statistical properties, such as periodicity, it will be hard for methods based on local pattern information to capture the underlying structure and, hence, afford generalization capabilities. Conversely, once this optimal representation is found, most of the problems may be solved by a linear method. Hence, the key is to find the proper representation. As a proof of concept we present a particular problem where the class distributions have a very intricate overlap on the space of original attributes. For this problem, the proposed algorithm finds a representation based on the trigonometric basis that provides a solution where some of the classical learning methods, e.g. multilayer perceptrons and decision trees, fail. The methodology is composed by a discrete search within the space of basis functions and a linear mapping performed by a Fisher discriminant. We play special emphasis on the first part. Finding the optimal combination of basis functions is a difficult problem because of its nongradient nature and the large number of possible combinations. We rely on the global search capabilities of a genetic algorithm to scan the space of function compositions.