Leveraging Transfer Learning and GAN Models for OCR from Engineering Documents

Abstract

Digital engineering, the digital transformation of engineering practice, is profoundly changing the traditional engineering practice towards the fast integration of digital technologies and digital models in the engineering processes' life cycles. The traditional engineering process heavily relies on static engineering documents (e.g., spreadsheets, technical drawings, and scanned documents) to store and share information across the engineering process. A critical task in digital engineering is to extract relevant textual information from traditional engineering documents into machine-readable and editable formats. This paper explores deep learning models and OCR methods to effectively extract textual information from engineering documents collected by the NAVY's military sealift command division. We propose a deep learning-based optical character recognition (OCR) framework for this task, which integrates several modules including a pre-trained text detection model, a fine-tuned OCR algorithm, and a deep generative model to augment data for the fine-tuning. Experimental results showed that the fine-tuning method significantly improved word accuracies of OCR models from 60%-70% to 90% and above. Furthermore, the deep adversarial generative approach had proved to be an effective model for data augmentation.