Chart-to-text generation using a hybrid deep network

• Published in: Advances in computational intelligence Vol. 3; no. 5; p. 19
• Main Authors: Wonglek, Nontaporn, Maneesinthu, Siriwalai, Srichaiyaperk, Sivakorn, Saengmuang, Teerapon, Siriborvornratanakul, Thitirat
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.10.2023; Springer Nature B.V
• Subjects: Artificial Intelligence; Automation; Borrowing; Charts; Computational Intelligence; Datasets; Deep learning; Descriptions; Education; Engineering; Expenditures; Format; Government spending; Keywords; Machine Learning; Metadata; Natural language; Natural language processing; Optical character recognition; Original Article; Text generation; Transformer; Optical character recognition; Data extraction; Chart to text
• ISSN: 2730-7794; 2730-7808
• DOI: 10.1007/s43674-023-00066-y

Text generation from charts is a task that involves automatically generating natural language text descriptions of data presented in chart form. This is a useful capability for tasks such as summarizing data for presentation or providing alternative representations of data for accessibility. In this work, we propose a hybrid deep network approach for text generation from table images in an academic format. The input to the model is a table image, which is first processed using Tesseract OCR (optical character recognition) to extract the data. The data are then passed through a Transformer (i.e., T5, K2T) model to generate the final text output. We evaluate the performance of our model on a dataset of academic papers. Results show that our network is able to generate high-quality text descriptions of charts. Specifically, the average BLEU scores are 0.072355 for T5 and 0.037907 for K2T. Our results demonstrate the effectiveness of the hybrid deep network approach for text generation from table images in an academic format.