Chart ScanX: An Automated Chart Analyzer

Abstract

The widespread use of charts and infographics as a means of data visualization in various domains has inspired recent research in automated chart understanding. There are various steps involved in the process of automated chart understanding and chart information extraction such as chart-type classification, text detection, text recognition and text-role classification. However, information extraction from chart images is a complex multitasked process due to style variations and, as a consequence, it is challenging to design an end-to-end system. In this project, we propose a deep learning-based framework that provides a solution for key steps in the chart information extraction pipeline. The proposed framework utilizes hierarchal vision transformers for the tasks of chart-type and text-role classification, while YOLOv7 for text detection. The detected text is then enhanced using Super Resolution Generative Adversarial Networks to improve the recognition output of the OCR. Experimental results on a benchmark dataset shows that our proposed framework outperforms SOTA in each step with F1-scores of 0.97 for chart-type classification, 0.91 for text-role classification, and a mean Average Precision of 0.95 for text detection