Deformation Estimation and Classification of Graphomotor Impressions - An Application to Neuropsychological Assessments

Abstract

Graphomotor skills of an individual can provide useful insight regarding his/her mental health and emotional state. Assessing these skills enables neuropsychologists to target areas of dysfunction and to design appropriate plans for rehabilitation. To assess graphomotor skills, clinical practitioners employ a variety of pen-and-paper based graphomotor tasks involving handwriting and drawings. Performance of an individual in these tasks is measured by using extensive scoring criteria that determines the presence/absence of various motor, perceptual, and cognitive deformations, by estimating deviations from expected stimulus. Nevertheless, manual scoring by human experts is time consuming and prone to inter-scorer bias. Computerized analysis of responses produced by at-risk subjects has high potential to address the aforementioned limitations of manual assessment. Furthermore, computerized analysis can also facilitate test standardization and validation, treatment efficacy assessment and disease progression monitoring. While a number of techniques are presented in the literature, there is a constant need to explore effective methods to translate domain knowledge into computational feature space. In this research, we propose a novel deformation modeling and estimating method, that can model a variety of visual-motor and visual-perceptual deformations from both online and offline samples of patient responses. The proposed scheme suggests feature extraction by means of pre-trained Convolutional Neural Networks (CNNs) for rich visual representation of samples of a particular deformation. By employing pre-trained ConvNets, we overcome the limitations of data scarcity and feature insufficiency that are common characteristics in this domain. To further enhance representation deformation-specific augmentation is employed. These enhanced visual features are then used to train classical machine learning classifiers to predict the presence or absence of particular deformation(s) in the test sample. The proposed method can be employed in a wide variety of scenarios to analyze a neuropsychological response. The performance of our proposed deformation estimation and classification method is evaluated in two empirical settings, that are popularly targeted by the relevant research community as well i.e. (a) Early detection of a neurodegenerative disease and (b) Scoring of a neuropsychological test with an extensive scoring standard. Our first scenario involves the identification of visual-motor deformations like tremor and micrographia from the graphomotor responses of elderly for the prediction of Parkinson’s Disease (PD). We employ a popular benchmark dataset ‘Parkinson’s Disease Handwriting (PaHaW)’ database, that comprises multiple graphomotor tasks performed by subjects suffering from PD and healthy controls (HC). To highlight fine imperfections caused due to associated motor dysfunctions (like tremor), we propose two non-linear transformations of the raw images using median and edge enhancing filters. For feature extraction, we employ the convolutional base of a pre-trained ConvNet and combine the extracted features of different representations to provide further enhancement. The combined feature vectors from each task are then employed to train a task-specific Support Vector Machine (SVM) classifier that predicts the response as belonging to either of the two classes (PD/HC). From our evaluations, it is observed that each task has a different impact on the classification accuracy. Due to this reason, decisions of i ii all tasks performed by a subject are combined by applying majority voting. The ensemble approach not only improved the overall classification results (83%) but also mitigated the negative impact of a task on the predictive potential of the extracted features. The second study targets the identification of eleven visual-perceptual deformations outlined in the Lacks’ scoring standard for the assessment of a Bender Gestalt Test (BGT) response. Perceptual deformations are challenging to model due to the insufficiency of features and reliance on extensive heuristics. We apply our proposed deformation modeling and classification method to identify Lacks’ eleven indicators of perceptual dysfunction from samples of children with learning difficulties. Due to lack of relevant datasets, a customized dataset is employed for the evaluation purposes. Unlike conventional sketch recognition, where intra-shape class variations are diminished and inter-shape class variations are enhanced, our proposed methodology enhances deformationspecific intra-shape class variations and generalizes inter-shape class similarities. This has not been attempted previously and enables the identification of same deformation across multiple shapes and different deformations with in same shape class. Once again, deformation-specific transformations are employed to ensure representation of the missing classes as well as to enrich features. Several combinations of pre-trained ConvNets and binary classifiers are assessed to determine the best combination. Results of our experiments show that best classification rates (i.e. mean accuracies ranging from 79.1% to 97.6%) are achieved across all deformations when features extracted from ResNet101 are used to train Linear Discriminant Analysis (LDA) classifier. Decisions from different deformation-specific classifiers are combined to quantify errors as required by the scoring standard. From the results of our experiments, we found that the nature of the deformation contributes the most in the performance of the classifier. This finding is coherent with that observed during manual scoring, as there exists greater inter-scorer difference for some deformations as compared to others. Nonetheless, the outcomes of both scenarios highlight the effectiveness of our proposed methodology in terms of reliability and robustness and support its potential for providing a solid basis for relevant end-to-end systems that can easily be integr