Deep Boltzmann Machines for i-Vector Based Audio-Visual Person Identification

• Published in: Image and Video Technology Vol. 9431; pp. 631 - 641
• Main Authors: Alam, Mohammad Rafiqul, Bennamoun, Mohammed, Togneri, Roberto, Sohel, Ferdous
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2016; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Boltzmann Machine; Deep Belief Network; Graphical & digital media applications; Graphics programming; Hide Layer; Image processing; Speaker Recognition; Universal Background Model
• ISBN: 9783319294506; 3319294504
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-29451-3_50

We propose an approach using DBM-DNNs for i-vector based audio-visual person identification. The unsupervised training of two Deep Boltzmann Machines DBM $$_{\text {speech}}$$ and DBM $$_\text {face}$$ is performed using unlabeled audio and visual data from a set of background subjects. The DBMs are then used to initialize two corresponding DNNs for classification, referred to as the DBM-DNN $$_{\text {speech}}$$ and DBM-DNN $$_{\text {face}}$$ in this paper. The DBM-DNNs are discriminatively fine-tuned using the back-propagation on a set of training data and evaluated on a set of test data from the target subjects. We compared their performance with the cosine distance (cosDist) and the state-of-the-art DBN-DNN classifier. We also tested three different configurations of the DBM-DNNs. We show that DBM-DNNs with two hidden layers and 800 units in each hidden layer achieved best identification performance for 400 dimensional i-vectors as input. Our experiments were carried out on the challenging MOBIO dataset.