Unconstrained Urdu Handwriting Recognition Using Transformers

Abstract

The technique is intended to overcome the difficulty of recognising Arabic-like scripts, particularly Urdu, which is a challenging and understudied script for offline handwriting recognition. The encoder-decoder approach the researchers suggest is attention-based and intended to interpret Urdu contextually. The transformer design, which has been successful in many natural language processing problems, is used in the model. In order to represent the contextdependent character forms, two-dimensional structure, gaps, overlaps, and diacritics found in Urdu script, it makes use of attention processes. The model may concentrate on pertinent sections of the input sequence using attention processes in order to produce precise predictions. A unique localization penalty is implemented to encourage the model to focus on one place at a time while identifying the next actor. This penalty acts as a regularisation term, directing the model to focus attention in one place rather than dispersing it across several. This improvement enables the model to more accurately comprehend and represent the unique traits and structure of the Urdu script. The researchers improve the only full and publicly accessible handwritten Urdu dataset in terms of ground-truth annotations in addition to creating the attentionbased model. This process makes sure the dataset is appropriately labelled and gives the model trustworthy training and assessment data. The model is tested on datasets in Urdu in order to gauge the effectiveness of the suggested technique. The results show that the pre-trained attention-based visual encoder and textual decoder transformer technique performs better at recognising Urdu scripts than straightforward attention-based transformer models. This shows that the localisation penalty and the attention-based encoder-decoder model, when used together, successfully solve the issues unique to Arabic-like scripts, notably the Urdu language.