Information Extraction from Text Intensive and Visually Rich Banking Documents

• Published in: Information processing & management Vol. 57; no. 6; p. 102361
• Main Authors: Oral, Berke, Emekligil, Erdem, Arslan, Seçil, Eryiǧit, Gülşen
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.11.2020; Elsevier Science Ltd
• Subjects: Algorithms; Banking; Banking Documents; Data integration; Data processing; Deep Learning; Extraction processes; Feature extraction; Information Extraction; Information processing; Information retrieval; Learning; Machine learning; Named Entity Recognition; NLP in Finance; Object recognition; Relation Extraction; Text Intensive Documents; Transfer of funds; Visually Rich Documents; Workflow; Banking Documents; NLP in Finance; Named Entity Recognition; Visually Rich Documents; Relation Extraction; Information Extraction; Text Intensive Documents; Deep Learning
• ISSN: 0306-4573; 1873-5371
• DOI: 10.1016/j.ipm.2020.102361

•First study using visual and textual information for deep-learning based information extraction on text-intensive and visually rich scanned documents•First study to investigate deep learning algorithms in banking document understanding•Automation of customer banking order documents reduced cycle times significantly•Investigated traditional and deep learning approaches in noisy text NER•Novel graph-based complex relation extraction algorithm outperforms previous methods•N-ary, nested, document-level, and previously indeterminate quantity of complex relations extracted successfully•Incorporating document layout information improves performances substantially Document types, where visual and textual information plays an important role in their analysis and understanding, pose a new and attractive area for information extraction research. Although cheques, invoices, and receipts have been studied in some previous multi-modal studies, banking documents present an unexplored area due to the naturalness of the text they possess in addition to their visual richness. This article presents the first study which uses visual and textual information for deep-learning based information extraction on text-intensive and visually rich scanned documents which are, in this instance, unstructured banking documents, or more precisely, money transfer orders. The impact of using different neural word representations (i.e., FastText, ELMo, and BERT) on IE subtasks (namely, named entity recognition and relation extraction stages), positional features of words on document images and auxiliary learning with some other tasks are investigated. The article proposes a new relation extraction algorithm based on graph factorization to solve the complex relation extraction problem where the relations within documents are n-ary, nested, document-level, and previously indeterminate in quantity. Our experiments revealed that the use of deep learning algorithms yielded around 10 percentage points improvement on the IE sub-tasks. The inclusion of word positional features yielded around 3 percentage points of improvement in some specific information fields. Similarly, our auxiliary learning experiments yielded around 2 percentage points of improvement on some information fields associated with the specific transaction type detected by our auxiliary task. The integration of the information extraction system into a real banking environment reduced cycle times substantially. When compared to the manual workflow, document processing pipeline shortened book-to-book money transfers to 10 minutes (from 29 min.) and electronic fund transfers (EFT) to 17 minutes (from 41 min.) respectively.