Multimodal Mathematical Expressions Recognition: Case of Speech and Handwriting

In this work, we propose to combine two modalities, handwriting and speech, to build a mathematical expression recognition system. Based on two sub-systems which process each modality, we explore various fusion methods to resolve ambiguities which naturally occur independently. The results that are...

Full description

Saved in:
Bibliographic Details
Published inHuman-Computer Interaction. Interaction Modalities and Techniques pp. 77 - 86
Main Authors Medjkoune, Sofiane, Mouchere, Harold, Petitrenaud, Simon, Viard-Gaudin, Christian
Format Book Chapter
LanguageEnglish
Published Berlin, Heidelberg Springer Berlin Heidelberg 2013
SeriesLecture Notes in Computer Science
Subjects
data fusion
graphical languages
handwriting
Multimodality
speech
Online AccessGet full text
ISBN9783642393297
3642393292
ISSN0302-9743
1611-3349
1611-3349
DOI10.1007/978-3-642-39330-3_9

Cover

More Information
Summary:In this work, we propose to combine two modalities, handwriting and speech, to build a mathematical expression recognition system. Based on two sub-systems which process each modality, we explore various fusion methods to resolve ambiguities which naturally occur independently. The results that are reported on the HAMEX bimodal database show an improvement with respect to a mono-modal based system.
ISBN:9783642393297
3642393292
ISSN:0302-9743
1611-3349
1611-3349
DOI:10.1007/978-3-642-39330-3_9