Detection of Visual Concepts and Annotation of Images Using Ensembles of Trees for Hierarchical Multi-Label Classification

• Published in: Recognizing Patterns in Signals, Speech, Images and Videos pp. 152 - 161
• Main Authors: Dimitrovski, Ivica, Kocev, Dragi, Loskovska, Suzana, Džeroski, Sašo
• Format: Book Chapter
• Language: English; Japanese
• Published: Berlin, Heidelberg Springer Berlin Heidelberg 2010
• Series: Lecture Notes in Computer Science
• Subjects: Equal Error Rate; Image Annotation; Local Binary Pattern; Random Forest; Sift Descriptor
• ISBN: 9783642177101; 3642177107
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-642-17711-8_16

In this paper, we present a hierarchical multi-label classification system for visual concepts detection and image annotation. Hierarchical multi-label classification (HMLC) is a variant of classification where an instance may belong to multiple classes at the same time and these classes/labels are organized in a hierarchy. The system is composed of two parts: feature extraction and classification/annotation. The feature extraction part provides global and local descriptions of the images. These descriptions are then used to learn a classifier and to annotate an image with the corresponding concepts. To this end, we use predictive clustering trees (PCTs), which are able to classify target concepts that are organized in a hierarchy. Our approach to HMLC exploits the annotation hierarchy by building a single predictive clustering tree that can simultaneously predict all of the labels used to annotate an image. Moreover, we constructed ensembles (random forests) of PCTs, to improve the predictive performance. We tested our system on the image database from the ImageCLEF@ICPR 2010 photo annotation task. The extensive experiments conducted on the benchmark database show that our system has very high predictive performance and can be easily scaled to large number of visual concepts and large amounts of data.