Multimodal Mathematical Expressions Recognition: Case of Speech and Handwriting - Medjkoune, Sofiane; Mouchere, Harold; Petitrenaud, Simon; Viard-Gaudin, Christian (Human-Computer Interaction. Interaction Modalities and Techniques, 2013)
In this work, we propose to combine two modalities, handwriting and speech, to build a mathematical expression recognition system. Based on two sub-systems which process each modality, we explore various fusion methods to resolve ambiguities which naturally occur independently. The results that are reported on the HAMEX bimodal database show an improvement with respect to a mono-modal based system.