Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition

Handwritten mathematical expression recognition is a challenging problem due to the complicated two-dimensional structures, ambiguous handwriting input and variant scales of handwritten math symbols. To settle this problem, recently we propose the attention based encoder-decoder model that recognize...

Full description

Saved in:
Bibliographic Details
Published in2018 24th International Conference on Pattern Recognition (ICPR) pp. 2245 - 2250
Main Authors Zhang, Jianshu, Du, Jun, Dai, Lirong
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.08.2018
Subjects
Cats
Convolution
Decoding
Feature extraction
Handwriting recognition
Mathematical model
Training
Online AccessGet full text
DOI10.1109/ICPR.2018.8546031

Cover

More Information
Summary:Handwritten mathematical expression recognition is a challenging problem due to the complicated two-dimensional structures, ambiguous handwriting input and variant scales of handwritten math symbols. To settle this problem, recently we propose the attention based encoder-decoder model that recognizes mathematical expression images from two-dimensional layouts to one-dimensional LaTeX strings. In this study, we improve the encoder by employing densely connected convolutional networks as they can strengthen feature extraction and facilitate gradient propagation especially on a small training set. We also present a novel multi-scale attention model which is employed to deal with the recognition of math symbols in different scales and restore the fine-grained details dropped by pooling operations. Validated on the CROHME competition task, the proposed method significantly outperforms the state-of-the-art methods with an expression recognition accuracy of 52.8% on CROHME 2014 and 50.1% on CROHME 2016, by only using the official training dataset.
DOI:10.1109/ICPR.2018.8546031