Learning Siamese Features for Finger Spelling Recognition

• Published in: Advanced Concepts for Intelligent Vision Systems Vol. 10617; pp. 225 - 236
• Main Authors: Kwolek, Bogdan, Sako, Shinji
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2017; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: CNNs; Finger spelling recognition; Siamese neural networks
• ISBN: 3319703528; 9783319703527
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-70353-4_20

This paper is devoted to finger spelling recognition on the basis of images acquired by a single color camera. The recognition is realized on the basis of learned low-dimensional embeddings. The embeddings are calculated both by single as well as multiple siamese-based convolutional neural networks. We train classifiers operating on such features as well as convolutional neural networks operating on raw images. The evaluations are performed on freely available dataset with finger spellings of Japanese Sign Language. The best results are achieved by a classifier trained on concatenated features of multiple siamese networks.